Login
[x]
Log in using an account from:
Fedora Account System
Red Hat Associate
Red Hat Customer
Or login using a Red Hat Bugzilla account
Forgot Password
Login:
Hide Forgot
Create an Account
Red Hat Bugzilla – Attachment 949986 Details for
Bug 1156119
Removing a ISCSI Domain fails with Internal Server Error - java.lang.ArrayIndexOutOfBoundsException
[?]
New
Simple Search
Advanced Search
My Links
Browse
Requests
Reports
Current State
Search
Tabular reports
Graphical reports
Duplicates
Other Reports
User Changes
Plotly Reports
Bug Status
Bug Severity
Non-Defaults
|
Product Dashboard
Help
Page Help!
Bug Writing Guidelines
What's new
Browser Support Policy
5.0.4.rh83 Release notes
FAQ
Guides index
User guide
Web Services
Contact
Legal
This site requires JavaScript to be enabled to function correctly, please enable it.
vdsm.log
vdsm_iscsi_failure.log (text/plain), 1.84 MB, created by
Carlos Mestre González
on 2014-10-23 15:26:47 UTC
(
hide
)
Description:
vdsm.log
Filename:
MIME Type:
Creator:
Carlos Mestre González
Created:
2014-10-23 15:26:47 UTC
Size:
1.84 MB
patch
obsolete
>Thread-49098::DEBUG::2014-10-23 16:46:24,604::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d3175b3f-77fc-48d8-ad6d-3980610b86a7`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:46:24,604::task::993::Storage.TaskManager.Task::(_decref) Task=`d3175b3f-77fc-48d8-ad6d-3980610b86a7`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:46:24,620::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:24,621::task::1191::Storage.TaskManager.Task::(prepare) Task=`d3175b3f-77fc-48d8-ad6d-3980610b86a7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:24,621::task::595::Storage.TaskManager.Task::(_updateState) Task=`d3175b3f-77fc-48d8-ad6d-3980610b86a7`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:46:24,622::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:46:24,622::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:46:24,623::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:46:24,624::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:46:24,625::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:46:24,625::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:46:24,627::task::993::Storage.TaskManager.Task::(_decref) Task=`d3175b3f-77fc-48d8-ad6d-3980610b86a7`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:46:26,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:26,235::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115909 s, 88.3 MB/s\n'; <rc> = 0 >Thread-49139::DEBUG::2014-10-23 16:46:26,425::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:46:26,427::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,427::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,427::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,428::__init__::232::IOProcess::(_processLogs) (50447) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:46:26,428::__init__::232::IOProcess::(_processLogs) (50447) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,428::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,428::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,429::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,429::__init__::232::IOProcess::(_processLogs) (50448) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:46:26,429::__init__::232::IOProcess::(_processLogs) (50448) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:26,430::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:46:26,431::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,431::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,431::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,431::__init__::232::IOProcess::(_processLogs) (50449) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:46:26,432::__init__::232::IOProcess::(_processLogs) (50449) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:26,432::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:46:26,433::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,433::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,433::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,433::__init__::232::IOProcess::(_processLogs) (50450) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:46:26,433::__init__::232::IOProcess::(_processLogs) (50450) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:26,433::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-41::DEBUG::2014-10-23 16:46:26,434::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,434::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,434::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,434::__init__::232::IOProcess::(_processLogs) (50451) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:26,435::__init__::232::IOProcess::(_processLogs) (50451) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,439::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,439::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,439::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,439::__init__::232::IOProcess::(_processLogs) (50452) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:46:26,440::__init__::232::IOProcess::(_processLogs) (50452) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:26,440::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49139::INFO::2014-10-23 16:46:26,440::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49139::INFO::2014-10-23 16:46:26,440::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:46:26,440::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,441::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,441::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,441::__init__::232::IOProcess::(_processLogs) (50453) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:26,441::__init__::232::IOProcess::(_processLogs) (50453) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:26,442::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:46:26,447::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000346639 s, 1.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:26,448::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,448::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,448::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,448::__init__::232::IOProcess::(_processLogs) (50454) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:26,448::__init__::232::IOProcess::(_processLogs) (50454) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,890::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,891::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,891::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,891::__init__::232::IOProcess::(_processLogs) (50455) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:26,891::__init__::232::IOProcess::(_processLogs) (50455) Queuing response >Thread-49080::DEBUG::2014-10-23 16:46:26,891::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:46:26,897::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000324598 s, 989 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:26,898::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,898::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,898::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,898::__init__::232::IOProcess::(_processLogs) (50456) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:26,899::__init__::232::IOProcess::(_processLogs) (50456) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,899::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,899::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,900::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,900::__init__::232::IOProcess::(_processLogs) (50457) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:26,900::__init__::232::IOProcess::(_processLogs) (50457) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) (50458) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) (50458) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:26,901::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:26,902::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:26,902::__init__::232::IOProcess::(_processLogs) (50459) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:26,902::__init__::232::IOProcess::(_processLogs) (50459) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:28,245::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:28,284::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115565 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:30,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:30,334::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115998 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:32,343::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:32,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116245 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:34,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:34,432::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117643 s, 87.0 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:46:34,721::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:46:34,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`a92c044e-7f46-470d-915e-67f0a1f7f592`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:46:34,721::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:46:34,734::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:34,734::task::1191::Storage.TaskManager.Task::(prepare) Task=`a92c044e-7f46-470d-915e-67f0a1f7f592`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:34,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`a92c044e-7f46-470d-915e-67f0a1f7f592`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:46:34,734::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:46:34,734::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:46:34,734::task::993::Storage.TaskManager.Task::(_decref) Task=`a92c044e-7f46-470d-915e-67f0a1f7f592`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:46:34,745::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:46:34,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:46:34,746::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:46:34,746::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`5684b128-ec05-4b3c-990a-a00525fc856f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:46:34,746::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:46:34,746::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:46:34,746::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`5684b128-ec05-4b3c-990a-a00525fc856f`::Granted request >Thread-49098::DEBUG::2014-10-23 16:46:34,746::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:46:34,747::task::993::Storage.TaskManager.Task::(_decref) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:46:34,759::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:34,759::task::1191::Storage.TaskManager.Task::(prepare) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:34,760::task::595::Storage.TaskManager.Task::(_updateState) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:46:34,760::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:46:34,760::task::993::Storage.TaskManager.Task::(_decref) Task=`c482b14b-6acc-49cb-b4be-9c45c1fb1a38`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:46:36,441::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-41::DEBUG::2014-10-23 16:46:36,450::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,450::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,451::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,451::__init__::232::IOProcess::(_processLogs) (50460) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:36,456::__init__::232::IOProcess::(_processLogs) (50460) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:36,459::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:46:36,466::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000441935 s, 1.0 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:36,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,473::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,473::__init__::232::IOProcess::(_processLogs) (50461) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:36,474::__init__::232::IOProcess::(_processLogs) (50461) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:36,503::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0320986 s, 31.9 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:36,903::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,903::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,903::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,903::__init__::232::IOProcess::(_processLogs) (50462) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:36,903::__init__::232::IOProcess::(_processLogs) (50462) Queuing response >Thread-49080::DEBUG::2014-10-23 16:46:36,904::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:46:36,909::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000322872 s, 994 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:36,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,910::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,910::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,911::__init__::232::IOProcess::(_processLogs) (50463) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:36,911::__init__::232::IOProcess::(_processLogs) (50463) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:36,911::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) (50464) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) (50464) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,912::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) (50465) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) (50465) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:36,913::__init__::232::IOProcess::(_processLogs) (50466) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:36,914::__init__::232::IOProcess::(_processLogs) (50466) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:38,513::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:38,553::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116688 s, 87.8 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:46:39,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`5803265d-e2d2-4d05-b6b9-bb5a4fce2a16`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:46:39,857::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:46:39,857::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441935', 'lastCheck': '3.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000322872', 'lastCheck': '2.9', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:46:39,857::task::1191::Storage.TaskManager.Task::(prepare) Task=`5803265d-e2d2-4d05-b6b9-bb5a4fce2a16`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000441935', 'lastCheck': '3.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000322872', 'lastCheck': '2.9', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:46:39,857::task::595::Storage.TaskManager.Task::(_updateState) Task=`5803265d-e2d2-4d05-b6b9-bb5a4fce2a16`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:46:39,857::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:46:39,857::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:46:39,858::task::993::Storage.TaskManager.Task::(_decref) Task=`5803265d-e2d2-4d05-b6b9-bb5a4fce2a16`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:46:40,562::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:40,602::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115581 s, 88.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:42,611::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:42,651::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117197 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:44,663::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:44,716::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0224856 s, 45.5 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:46:44,804::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:46:44,806::task::595::Storage.TaskManager.Task::(_updateState) Task=`f623c8d1-5cf6-4464-89ae-c6bfb64059f2`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:46:44,807::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:46:44,820::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:44,820::task::1191::Storage.TaskManager.Task::(prepare) Task=`f623c8d1-5cf6-4464-89ae-c6bfb64059f2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:44,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`f623c8d1-5cf6-4464-89ae-c6bfb64059f2`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:46:44,820::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:46:44,820::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:46:44,820::task::993::Storage.TaskManager.Task::(_decref) Task=`f623c8d1-5cf6-4464-89ae-c6bfb64059f2`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:46:44,832::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:46:44,832::task::595::Storage.TaskManager.Task::(_updateState) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:46:44,832::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:46:44,833::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7c50bcd3-f071-40e1-880b-d01ff13d7db9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:46:44,833::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:46:44,833::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:46:44,833::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7c50bcd3-f071-40e1-880b-d01ff13d7db9`::Granted request >Thread-49098::DEBUG::2014-10-23 16:46:44,833::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:46:44,833::task::993::Storage.TaskManager.Task::(_decref) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:46:44,846::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:44,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:44,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:46:44,846::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:46:44,847::task::993::Storage.TaskManager.Task::(_decref) Task=`1670e7e3-a107-47a6-9ab4-5479fff74b6f`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:46:46,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,479::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,479::__init__::232::IOProcess::(_processLogs) (50467) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:46,480::__init__::232::IOProcess::(_processLogs) (50467) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:46,480::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:46:46,486::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000344116 s, 1.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:46,487::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,488::__init__::232::IOProcess::(_processLogs) (50468) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:46,488::__init__::232::IOProcess::(_processLogs) (50468) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:46,821::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:46,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011572 s, 88.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:46,915::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,915::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,915::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,915::__init__::232::IOProcess::(_processLogs) (50469) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:46,915::__init__::232::IOProcess::(_processLogs) (50469) Queuing response >Thread-49080::DEBUG::2014-10-23 16:46:46,916::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:46:46,921::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000323543 s, 992 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:46,922::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,922::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,923::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,923::__init__::232::IOProcess::(_processLogs) (50470) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:46,923::__init__::232::IOProcess::(_processLogs) (50470) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) (50471) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) (50471) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:46,924::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) (50472) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) (50472) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:46,925::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:46,926::__init__::232::IOProcess::(_processLogs) (50473) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:46,926::__init__::232::IOProcess::(_processLogs) (50473) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:48,870::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:48,910::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116401 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:50,919::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:50,959::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115999 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:52,969::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:53,009::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115653 s, 88.5 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:46:54,878::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:46:54,878::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dd1bc31-e5da-4ddb-a5ad-4f0aae6abb33`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:46:54,878::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:46:54,890::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:54,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`0dd1bc31-e5da-4ddb-a5ad-4f0aae6abb33`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:46:54,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`0dd1bc31-e5da-4ddb-a5ad-4f0aae6abb33`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:46:54,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:46:54,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:46:54,890::task::993::Storage.TaskManager.Task::(_decref) Task=`0dd1bc31-e5da-4ddb-a5ad-4f0aae6abb33`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:46:54,902::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:46:54,903::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:46:54,903::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:46:54,903::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`8bfc03bd-86a8-442f-bbda-0d1bcd4b05ae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:46:54,903::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:46:54,903::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:46:54,903::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`8bfc03bd-86a8-442f-bbda-0d1bcd4b05ae`::Granted request >Thread-49098::DEBUG::2014-10-23 16:46:54,903::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:46:54,904::task::993::Storage.TaskManager.Task::(_decref) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:46:54,916::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:54,916::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:46:54,916::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:46:54,916::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:46:54,916::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:46:54,916::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:46:54,917::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:46:54,917::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:46:54,917::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:46:54,917::task::993::Storage.TaskManager.Task::(_decref) Task=`ea4f19e0-c0ff-4b66-9cfa-ab18db5bd3b3`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:46:55,018::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:55,058::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116509 s, 87.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:46:55,108::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b1a32d5-956a-497d-ab8d-b5283b311759`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:46:55,108::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:46:55,109::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000344116', 'lastCheck': '8.6', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000323543', 'lastCheck': '8.2', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:46:55,109::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b1a32d5-956a-497d-ab8d-b5283b311759`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000344116', 'lastCheck': '8.6', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000323543', 'lastCheck': '8.2', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:46:55,109::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b1a32d5-956a-497d-ab8d-b5283b311759`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:46:55,109::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:46:55,109::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:46:55,109::task::993::Storage.TaskManager.Task::(_decref) Task=`7b1a32d5-956a-497d-ab8d-b5283b311759`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:46:56,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,489::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,489::__init__::232::IOProcess::(_processLogs) (50474) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:56,489::__init__::232::IOProcess::(_processLogs) (50474) Queuing response >Thread-49139::DEBUG::2014-10-23 16:46:56,490::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:46:56,496::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000330074 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:56,497::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,497::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,497::__init__::232::IOProcess::(_processLogs) (50475) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:56,497::__init__::232::IOProcess::(_processLogs) (50475) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:56,927::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,927::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,927::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,927::__init__::232::IOProcess::(_processLogs) (50476) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:56,927::__init__::232::IOProcess::(_processLogs) (50476) Queuing response >Thread-49080::DEBUG::2014-10-23 16:46:56,928::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:46:56,934::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000372288 s, 862 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:46:56,934::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,934::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,935::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,935::__init__::232::IOProcess::(_processLogs) (50477) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:46:56,935::__init__::232::IOProcess::(_processLogs) (50477) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) (50478) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) (50478) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,936::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) (50479) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) (50479) Queuing response >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:46:56,937::__init__::232::IOProcess::(_processLogs) (50480) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:46:56,938::__init__::232::IOProcess::(_processLogs) (50480) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:46:57,067::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:57,107::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115122 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:46:59,116::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:46:59,156::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116323 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:01,165::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:01,206::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116474 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:03,215::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:03,255::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116825 s, 87.7 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:04,949::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:47:04,949::task::595::Storage.TaskManager.Task::(_updateState) Task=`b99912a3-a3f4-4393-9b51-adb2d1a554a1`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:04,949::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:04,961::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:04,961::task::1191::Storage.TaskManager.Task::(prepare) Task=`b99912a3-a3f4-4393-9b51-adb2d1a554a1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:04,961::task::595::Storage.TaskManager.Task::(_updateState) Task=`b99912a3-a3f4-4393-9b51-adb2d1a554a1`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:04,961::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:04,961::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:04,961::task::993::Storage.TaskManager.Task::(_decref) Task=`b99912a3-a3f4-4393-9b51-adb2d1a554a1`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:04,972::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:47:04,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`931b29e2-077f-472f-8c16-3594b6667995`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:04,972::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:04,972::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`dae59e69-e48e-441b-ad4b-041cb421f7a2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:04,973::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:04,973::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:04,973::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`dae59e69-e48e-441b-ad4b-041cb421f7a2`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:04,973::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`931b29e2-077f-472f-8c16-3594b6667995`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:04,973::task::993::Storage.TaskManager.Task::(_decref) Task=`931b29e2-077f-472f-8c16-3594b6667995`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:04,985::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:04,985::task::1191::Storage.TaskManager.Task::(prepare) Task=`931b29e2-077f-472f-8c16-3594b6667995`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:04,985::task::595::Storage.TaskManager.Task::(_updateState) Task=`931b29e2-077f-472f-8c16-3594b6667995`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:04,985::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:04,986::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:04,986::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:04,986::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:04,986::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:04,986::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:04,986::task::993::Storage.TaskManager.Task::(_decref) Task=`931b29e2-077f-472f-8c16-3594b6667995`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:05,264::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:05,304::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011538 s, 88.8 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:06,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,499::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,499::__init__::232::IOProcess::(_processLogs) (50481) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:06,499::__init__::232::IOProcess::(_processLogs) (50481) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:06,500::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:06,511::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000309532 s, 1.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:06,512::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,512::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,512::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,512::__init__::232::IOProcess::(_processLogs) (50482) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:06,512::__init__::232::IOProcess::(_processLogs) (50482) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:06,939::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,939::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,939::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,939::__init__::232::IOProcess::(_processLogs) (50483) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:06,939::__init__::232::IOProcess::(_processLogs) (50483) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:06,940::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:06,946::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000330887 s, 970 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:06,947::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,947::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,947::__init__::232::IOProcess::(_processLogs) (50484) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:06,947::__init__::232::IOProcess::(_processLogs) (50484) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:06,948::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,948::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,948::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,948::__init__::232::IOProcess::(_processLogs) (50485) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:06,948::__init__::232::IOProcess::(_processLogs) (50485) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:06,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,949::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,949::__init__::232::IOProcess::(_processLogs) (50486) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:06,949::__init__::232::IOProcess::(_processLogs) (50486) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:06,950::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:06,950::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:06,950::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:06,950::__init__::232::IOProcess::(_processLogs) (50487) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:06,950::__init__::232::IOProcess::(_processLogs) (50487) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:07,313::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:07,353::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115124 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:09,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:09,402::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115464 s, 88.7 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:10,310::task::595::Storage.TaskManager.Task::(_updateState) Task=`e27fdadc-4987-44be-a11c-964fa9b4d761`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:10,310::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:47:10,310::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000309532', 'lastCheck': '3.8', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000330887', 'lastCheck': '3.4', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:10,310::task::1191::Storage.TaskManager.Task::(prepare) Task=`e27fdadc-4987-44be-a11c-964fa9b4d761`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000309532', 'lastCheck': '3.8', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000330887', 'lastCheck': '3.4', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:10,310::task::595::Storage.TaskManager.Task::(_updateState) Task=`e27fdadc-4987-44be-a11c-964fa9b4d761`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:10,310::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:10,310::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:10,310::task::993::Storage.TaskManager.Task::(_decref) Task=`e27fdadc-4987-44be-a11c-964fa9b4d761`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:11,411::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:11,455::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0156937 s, 65.2 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:13,464::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:13,504::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116493 s, 87.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:15,026::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [49e5bf0b] >Thread-49099::DEBUG::2014-10-23 16:47:15,026::task::595::Storage.TaskManager.Task::(_updateState) Task=`682f1b4b-35bf-4753-817f-94184ec89d2f`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:15,026::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:15,037::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:15,038::task::1191::Storage.TaskManager.Task::(prepare) Task=`682f1b4b-35bf-4753-817f-94184ec89d2f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:15,038::task::595::Storage.TaskManager.Task::(_updateState) Task=`682f1b4b-35bf-4753-817f-94184ec89d2f`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:15,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:15,038::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:15,038::task::993::Storage.TaskManager.Task::(_decref) Task=`682f1b4b-35bf-4753-817f-94184ec89d2f`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:15,050::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [49e5bf0b] >Thread-49098::DEBUG::2014-10-23 16:47:15,050::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:15,050::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:15,050::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a2fd9b81-889c-4ba2-9cc9-5b26bcbb78a9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:15,050::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:15,050::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:15,051::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a2fd9b81-889c-4ba2-9cc9-5b26bcbb78a9`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:15,051::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:15,051::task::993::Storage.TaskManager.Task::(_decref) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:15,063::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:15,063::task::1191::Storage.TaskManager.Task::(prepare) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:15,063::task::595::Storage.TaskManager.Task::(_updateState) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:15,063::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:15,063::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:15,064::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:15,064::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:15,064::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:15,064::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:15,064::task::993::Storage.TaskManager.Task::(_decref) Task=`0cc650c4-4a5a-44f9-82c3-2c6c8c2f9394`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:15,514::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:15,554::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117496 s, 87.2 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:16,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,513::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,514::__init__::232::IOProcess::(_processLogs) (50488) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:16,514::__init__::232::IOProcess::(_processLogs) (50488) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:16,514::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:16,521::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000355336 s, 1.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:16,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,522::__init__::232::IOProcess::(_processLogs) (50489) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:16,522::__init__::232::IOProcess::(_processLogs) (50489) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:16,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,952::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,952::__init__::232::IOProcess::(_processLogs) (50490) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:16,952::__init__::232::IOProcess::(_processLogs) (50490) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:16,952::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:16,958::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000320692 s, 1.0 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:16,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,959::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,959::__init__::232::IOProcess::(_processLogs) (50491) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:16,959::__init__::232::IOProcess::(_processLogs) (50491) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:16,960::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,960::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,960::__init__::232::IOProcess::(_processLogs) (50492) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) (50492) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) (50493) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:16,961::__init__::232::IOProcess::(_processLogs) (50493) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:16,962::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:16,962::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:16,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:16,962::__init__::232::IOProcess::(_processLogs) (50494) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:16,962::__init__::232::IOProcess::(_processLogs) (50494) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:17,563::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:17,616::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219015 s, 46.8 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:19,678::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:19,719::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011857 s, 86.4 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:21,728::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:21,769::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011794 s, 86.8 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:23,778::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:23,819::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117773 s, 86.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:25,142::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:47:25,142::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5ad3b43-7935-4f2d-8ccf-a38976917e8f`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:25,142::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:25,154::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:25,154::task::1191::Storage.TaskManager.Task::(prepare) Task=`e5ad3b43-7935-4f2d-8ccf-a38976917e8f`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:25,154::task::595::Storage.TaskManager.Task::(_updateState) Task=`e5ad3b43-7935-4f2d-8ccf-a38976917e8f`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:25,154::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:25,154::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:25,154::task::993::Storage.TaskManager.Task::(_decref) Task=`e5ad3b43-7935-4f2d-8ccf-a38976917e8f`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:25,165::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:47:25,166::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:25,166::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:25,166::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`fa308170-e95c-4b0a-af23-38e47b05bc93`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:25,166::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:25,166::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:25,166::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`fa308170-e95c-4b0a-af23-38e47b05bc93`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:25,166::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:25,167::task::993::Storage.TaskManager.Task::(_decref) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:25,179::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:25,179::task::1191::Storage.TaskManager.Task::(prepare) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:25,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:25,179::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:25,179::task::993::Storage.TaskManager.Task::(_decref) Task=`cf840c6b-9202-43bf-bd2c-ef1ad184f1d6`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:47:25,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8ee2696-2d04-4b0b-9755-6c4394994fb5`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:25,521::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:47:25,521::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355336', 'lastCheck': '9.0', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000320692', 'lastCheck': '8.6', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:25,521::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8ee2696-2d04-4b0b-9755-6c4394994fb5`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000355336', 'lastCheck': '9.0', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000320692', 'lastCheck': '8.6', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:25,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8ee2696-2d04-4b0b-9755-6c4394994fb5`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:25,521::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:25,521::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:25,521::task::993::Storage.TaskManager.Task::(_decref) Task=`a8ee2696-2d04-4b0b-9755-6c4394994fb5`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:25,828::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:25,868::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117068 s, 87.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:26,523::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,523::__init__::232::IOProcess::(_processLogs) (50495) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:26,524::__init__::232::IOProcess::(_processLogs) (50495) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:26,524::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:26,530::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000334201 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:26,531::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,531::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,531::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,531::__init__::232::IOProcess::(_processLogs) (50496) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:26,531::__init__::232::IOProcess::(_processLogs) (50496) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:26,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,964::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,964::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,964::__init__::232::IOProcess::(_processLogs) (50497) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:26,964::__init__::232::IOProcess::(_processLogs) (50497) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:26,965::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:26,971::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000321358 s, 999 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:26,971::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,971::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,972::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,972::__init__::232::IOProcess::(_processLogs) (50498) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:26,972::__init__::232::IOProcess::(_processLogs) (50498) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) (50499) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) (50499) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:26,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) (50500) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) (50500) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:26,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:26,975::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:26,975::__init__::232::IOProcess::(_processLogs) (50501) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:26,975::__init__::232::IOProcess::(_processLogs) (50501) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:27,878::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:27,918::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117319 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:29,927::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:29,967::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116333 s, 88.0 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:31,976::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:32,016::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117364 s, 87.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:34,026::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:34,066::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116019 s, 88.3 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:35,284::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:47:35,284::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c5660f4-0f33-4ffd-bb8d-3c0a43f5b728`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:35,284::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:35,296::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:35,297::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c5660f4-0f33-4ffd-bb8d-3c0a43f5b728`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:35,297::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c5660f4-0f33-4ffd-bb8d-3c0a43f5b728`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:35,297::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:35,297::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:35,297::task::993::Storage.TaskManager.Task::(_decref) Task=`3c5660f4-0f33-4ffd-bb8d-3c0a43f5b728`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:35,308::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:47:35,309::task::595::Storage.TaskManager.Task::(_updateState) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:35,309::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:35,309::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`953970cb-b2c3-4f1f-b6ed-ea08c95a23b0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:35,309::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:35,310::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:35,310::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`953970cb-b2c3-4f1f-b6ed-ea08c95a23b0`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:35,310::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:35,310::task::993::Storage.TaskManager.Task::(_decref) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:35,322::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:35,322::task::1191::Storage.TaskManager.Task::(prepare) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:35,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:35,322::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:35,323::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:35,323::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:35,323::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:35,323::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:35,323::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:35,323::task::993::Storage.TaskManager.Task::(_decref) Task=`699b253c-cc8e-4f59-a27e-01cda0ad651f`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:36,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:36,115::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115933 s, 88.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:36,533::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,533::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,533::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,533::__init__::232::IOProcess::(_processLogs) (50502) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:36,533::__init__::232::IOProcess::(_processLogs) (50502) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:36,534::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:36,540::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.00029786 s, 1.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:36,540::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,540::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,541::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,541::__init__::232::IOProcess::(_processLogs) (50503) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:36,541::__init__::232::IOProcess::(_processLogs) (50503) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:36,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,977::__init__::232::IOProcess::(_processLogs) (50504) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:36,977::__init__::232::IOProcess::(_processLogs) (50504) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:36,977::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:36,983::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000341118 s, 941 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:36,984::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,984::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,984::__init__::232::IOProcess::(_processLogs) (50505) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:36,984::__init__::232::IOProcess::(_processLogs) (50505) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:36,985::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,985::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,985::__init__::232::IOProcess::(_processLogs) (50506) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:36,985::__init__::232::IOProcess::(_processLogs) (50506) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:36,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,986::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,986::__init__::232::IOProcess::(_processLogs) (50507) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:36,986::__init__::232::IOProcess::(_processLogs) (50507) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:36,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:36,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:36,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:36,987::__init__::232::IOProcess::(_processLogs) (50508) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:36,987::__init__::232::IOProcess::(_processLogs) (50508) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:38,124::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:38,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116506 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:40,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:40,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011611 s, 88.2 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:40,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`342bc86d-4d02-4dab-a94e-c3d89b06e77f`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:40,755::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:47:40,755::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00029786', 'lastCheck': '4.2', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000341118', 'lastCheck': '3.8', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:40,755::task::1191::Storage.TaskManager.Task::(prepare) Task=`342bc86d-4d02-4dab-a94e-c3d89b06e77f`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00029786', 'lastCheck': '4.2', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000341118', 'lastCheck': '3.8', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:40,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`342bc86d-4d02-4dab-a94e-c3d89b06e77f`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:40,755::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:40,755::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:40,755::task::993::Storage.TaskManager.Task::(_decref) Task=`342bc86d-4d02-4dab-a94e-c3d89b06e77f`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:42,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:42,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116915 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:44,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:44,312::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116793 s, 87.7 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:45,414::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:47:45,414::task::595::Storage.TaskManager.Task::(_updateState) Task=`198908c2-0417-465e-be7a-1844e12e06fb`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:45,414::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:45,425::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:45,426::task::1191::Storage.TaskManager.Task::(prepare) Task=`198908c2-0417-465e-be7a-1844e12e06fb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:45,426::task::595::Storage.TaskManager.Task::(_updateState) Task=`198908c2-0417-465e-be7a-1844e12e06fb`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:45,426::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:45,426::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:45,426::task::993::Storage.TaskManager.Task::(_decref) Task=`198908c2-0417-465e-be7a-1844e12e06fb`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:45,437::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:47:45,437::task::595::Storage.TaskManager.Task::(_updateState) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:45,438::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:45,438::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`43847819-7574-42b7-9bca-ed9889573960`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:45,438::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:45,438::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:45,438::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`43847819-7574-42b7-9bca-ed9889573960`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:45,439::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:45,439::task::993::Storage.TaskManager.Task::(_decref) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:45,451::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:45,451::task::1191::Storage.TaskManager.Task::(prepare) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:45,451::task::595::Storage.TaskManager.Task::(_updateState) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:45,451::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:45,451::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:45,451::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:45,451::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:45,451::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:45,452::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:45,452::task::993::Storage.TaskManager.Task::(_decref) Task=`c094bac3-f2e6-4e7c-a05d-8fc9de3dcdab`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:47:46,322::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:46,362::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116284 s, 88.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:46,542::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,542::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,542::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,542::__init__::232::IOProcess::(_processLogs) (50509) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:46,543::__init__::232::IOProcess::(_processLogs) (50509) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:46,544::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:46,550::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000340728 s, 1.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:46,550::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,550::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,551::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,551::__init__::232::IOProcess::(_processLogs) (50510) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:46,551::__init__::232::IOProcess::(_processLogs) (50510) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:46,988::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,988::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,989::__init__::232::IOProcess::(_processLogs) (50511) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:46,989::__init__::232::IOProcess::(_processLogs) (50511) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:46,989::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:46,995::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000404154 s, 794 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:46,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,996::__init__::232::IOProcess::(_processLogs) (50512) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:46,996::__init__::232::IOProcess::(_processLogs) (50512) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:46,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,997::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) (50513) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) (50513) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,998::__init__::232::IOProcess::(_processLogs) (50514) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) (50514) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) (50515) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:46,999::__init__::232::IOProcess::(_processLogs) (50515) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:48,371::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:48,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0189579 s, 54.0 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:50,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:50,527::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116274 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:52,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:52,577::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116959 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:47:54,586::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:54,626::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115978 s, 88.3 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:47:55,499::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:47:55,499::task::595::Storage.TaskManager.Task::(_updateState) Task=`9929f018-68ce-4982-a11d-7113c01bfd67`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:55,499::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:47:55,510::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:55,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`9929f018-68ce-4982-a11d-7113c01bfd67`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:47:55,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`9929f018-68ce-4982-a11d-7113c01bfd67`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:55,511::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:55,511::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:55,511::task::993::Storage.TaskManager.Task::(_decref) Task=`9929f018-68ce-4982-a11d-7113c01bfd67`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:47:55,522::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:47:55,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:47:55,523::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:47:55,523::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`be6d581a-cfc9-415f-a56d-eb23bfb5bcbd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:47:55,523::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:47:55,523::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:47:55,523::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`be6d581a-cfc9-415f-a56d-eb23bfb5bcbd`::Granted request >Thread-49098::DEBUG::2014-10-23 16:47:55,524::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:47:55,524::task::993::Storage.TaskManager.Task::(_decref) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:47:55,536::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:55,536::task::1191::Storage.TaskManager.Task::(prepare) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:47:55,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:47:55,536::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:47:55,536::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:47:55,536::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:47:55,536::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:47:55,537::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:47:55,537::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:47:55,537::task::993::Storage.TaskManager.Task::(_decref) Task=`5cee0e3a-1583-46e5-a473-c7456f489aeb`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:47:55,921::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c2ad0bc-06aa-41fe-a250-941ff889de05`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:47:55,921::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:47:55,921::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000340728', 'lastCheck': '9.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404154', 'lastCheck': '8.9', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:55,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c2ad0bc-06aa-41fe-a250-941ff889de05`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000340728', 'lastCheck': '9.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000404154', 'lastCheck': '8.9', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:47:55,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c2ad0bc-06aa-41fe-a250-941ff889de05`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:47:55,922::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:47:55,922::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:47:55,922::task::993::Storage.TaskManager.Task::(_decref) Task=`9c2ad0bc-06aa-41fe-a250-941ff889de05`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:47:56,552::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:56,552::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:56,552::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:56,552::__init__::232::IOProcess::(_processLogs) (50516) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:56,553::__init__::232::IOProcess::(_processLogs) (50516) Queuing response >Thread-49139::DEBUG::2014-10-23 16:47:56,553::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:47:56,559::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.00031487 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:56,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:56,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:56,560::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:56,560::__init__::232::IOProcess::(_processLogs) (50517) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:56,560::__init__::232::IOProcess::(_processLogs) (50517) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:56,635::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:56,675::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117079 s, 87.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:57,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:57,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:57,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:57,001::__init__::232::IOProcess::(_processLogs) (50518) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:57,001::__init__::232::IOProcess::(_processLogs) (50518) Queuing response >Thread-49080::DEBUG::2014-10-23 16:47:57,002::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:47:57,007::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000322758 s, 995 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:47:57,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:57,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:57,008::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:57,008::__init__::232::IOProcess::(_processLogs) (50519) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:47:57,008::__init__::232::IOProcess::(_processLogs) (50519) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:57,009::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:57,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:57,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:57,009::__init__::232::IOProcess::(_processLogs) (50520) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) (50520) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) (50521) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:57,010::__init__::232::IOProcess::(_processLogs) (50521) Queuing response >Thread-41::DEBUG::2014-10-23 16:47:57,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:47:57,011::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:47:57,011::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:47:57,011::__init__::232::IOProcess::(_processLogs) (50522) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:47:57,011::__init__::232::IOProcess::(_processLogs) (50522) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:47:58,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:47:58,725::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116494 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:00,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:00,774::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116847 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:02,783::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:02,823::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116508 s, 87.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:04,832::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:04,873::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116357 s, 88.0 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:05,579::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:05,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b7bc9b3-9089-46df-9b3c-ef7229f7a4a2`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:05,579::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:05,591::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:05,591::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b7bc9b3-9089-46df-9b3c-ef7229f7a4a2`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:05,591::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b7bc9b3-9089-46df-9b3c-ef7229f7a4a2`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:05,591::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:05,591::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:05,591::task::993::Storage.TaskManager.Task::(_decref) Task=`8b7bc9b3-9089-46df-9b3c-ef7229f7a4a2`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:05,603::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:05,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:05,604::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:05,604::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9790b1b7-76d8-49e2-bf0d-ed6869b26a52`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:05,604::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:05,604::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:05,604::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9790b1b7-76d8-49e2-bf0d-ed6869b26a52`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:05,604::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:05,605::task::993::Storage.TaskManager.Task::(_decref) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:05,617::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:05,617::task::1191::Storage.TaskManager.Task::(prepare) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:05,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:05,617::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:05,618::task::993::Storage.TaskManager.Task::(_decref) Task=`57cc4ef1-6dc6-4c94-9e61-0428076283ab`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:06,562::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:06,562::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:06,562::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:06,562::__init__::232::IOProcess::(_processLogs) (50523) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:06,562::__init__::232::IOProcess::(_processLogs) (50523) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:06,563::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:06,569::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000334527 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:06,569::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:06,569::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:06,570::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:06,570::__init__::232::IOProcess::(_processLogs) (50524) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:06,570::__init__::232::IOProcess::(_processLogs) (50524) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:06,882::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:06,922::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116992 s, 87.5 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:07,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:07,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:07,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:07,013::__init__::232::IOProcess::(_processLogs) (50525) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:07,013::__init__::232::IOProcess::(_processLogs) (50525) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:07,013::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:07,020::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000324952 s, 988 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:07,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:07,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:07,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:07,021::__init__::232::IOProcess::(_processLogs) (50526) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:07,021::__init__::232::IOProcess::(_processLogs) (50526) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:07,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:07,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:07,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:07,022::__init__::232::IOProcess::(_processLogs) (50527) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:07,022::__init__::232::IOProcess::(_processLogs) (50527) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) (50528) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) (50528) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:07,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:07,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:07,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:07,024::__init__::232::IOProcess::(_processLogs) (50529) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:07,024::__init__::232::IOProcess::(_processLogs) (50529) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:08,931::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:08,971::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117128 s, 87.4 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:10,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:11,020::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116604 s, 87.8 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:11,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdf6b06a-cfa2-439a-bbff-b9d2c09417c6`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:11,179::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:48:11,180::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000334527', 'lastCheck': '4.6', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000324952', 'lastCheck': '4.2', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:11,180::task::1191::Storage.TaskManager.Task::(prepare) Task=`cdf6b06a-cfa2-439a-bbff-b9d2c09417c6`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000334527', 'lastCheck': '4.6', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000324952', 'lastCheck': '4.2', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:11,180::task::595::Storage.TaskManager.Task::(_updateState) Task=`cdf6b06a-cfa2-439a-bbff-b9d2c09417c6`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:11,180::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:11,180::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:11,180::task::993::Storage.TaskManager.Task::(_decref) Task=`cdf6b06a-cfa2-439a-bbff-b9d2c09417c6`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:48:13,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:13,069::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116296 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:15,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:15,119::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116483 s, 87.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:15,724::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:15,724::task::595::Storage.TaskManager.Task::(_updateState) Task=`11b8fe4c-f13e-4053-a3d2-edf7017420e1`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:15,724::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:15,740::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:15,741::task::1191::Storage.TaskManager.Task::(prepare) Task=`11b8fe4c-f13e-4053-a3d2-edf7017420e1`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:15,741::task::595::Storage.TaskManager.Task::(_updateState) Task=`11b8fe4c-f13e-4053-a3d2-edf7017420e1`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:15,741::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:15,741::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:15,741::task::993::Storage.TaskManager.Task::(_decref) Task=`11b8fe4c-f13e-4053-a3d2-edf7017420e1`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:15,752::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:15,752::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:15,752::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:15,752::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d4df6eb2-4e81-4094-9f3f-02397a12305c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:15,752::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:15,752::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:15,753::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d4df6eb2-4e81-4094-9f3f-02397a12305c`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:15,753::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:15,753::task::993::Storage.TaskManager.Task::(_decref) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:15,765::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:15,765::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:15,765::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:15,765::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:15,766::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:15,766::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:15,766::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:15,766::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:15,766::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:15,766::task::993::Storage.TaskManager.Task::(_decref) Task=`3c1bac17-12cf-490c-899a-625e9c659729`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:16,571::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:16,571::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:16,571::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:16,572::__init__::232::IOProcess::(_processLogs) (50530) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:16,572::__init__::232::IOProcess::(_processLogs) (50530) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:16,572::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:16,578::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000331995 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:16,579::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:16,579::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:16,579::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:16,579::__init__::232::IOProcess::(_processLogs) (50531) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:16,579::__init__::232::IOProcess::(_processLogs) (50531) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:17,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:17,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:17,025::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:17,026::__init__::232::IOProcess::(_processLogs) (50532) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:17,026::__init__::232::IOProcess::(_processLogs) (50532) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:17,026::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:17,032::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000303805 s, 1.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:17,033::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:17,033::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:17,033::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:17,033::__init__::232::IOProcess::(_processLogs) (50533) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:17,033::__init__::232::IOProcess::(_processLogs) (50533) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:17,034::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:17,034::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:17,034::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:17,034::__init__::232::IOProcess::(_processLogs) (50534) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:17,034::__init__::232::IOProcess::(_processLogs) (50534) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:17,035::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:17,035::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:17,035::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:17,035::__init__::232::IOProcess::(_processLogs) (50535) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:17,035::__init__::232::IOProcess::(_processLogs) (50535) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:17,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:17,036::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:17,036::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:17,036::__init__::232::IOProcess::(_processLogs) (50536) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:17,036::__init__::232::IOProcess::(_processLogs) (50536) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:17,128::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:17,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116879 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:19,178::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:19,218::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116701 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:21,227::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:21,274::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0180326 s, 56.8 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:23,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:23,323::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011685 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:25,332::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:25,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011532 s, 88.8 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:25,809::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:25,810::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2d5dc21-6b7b-447f-8183-7e3f83891f31`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:25,810::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:25,821::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:25,821::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2d5dc21-6b7b-447f-8183-7e3f83891f31`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:25,821::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2d5dc21-6b7b-447f-8183-7e3f83891f31`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:25,821::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:25,821::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:25,822::task::993::Storage.TaskManager.Task::(_decref) Task=`b2d5dc21-6b7b-447f-8183-7e3f83891f31`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:25,833::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:25,833::task::595::Storage.TaskManager.Task::(_updateState) Task=`049ad723-3205-4077-9bec-57af09b99053`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:25,833::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:25,834::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`81bbccfe-0246-49bf-a567-9838e682e3dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:25,834::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:25,834::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:25,834::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`81bbccfe-0246-49bf-a567-9838e682e3dd`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:25,834::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`049ad723-3205-4077-9bec-57af09b99053`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:25,834::task::993::Storage.TaskManager.Task::(_decref) Task=`049ad723-3205-4077-9bec-57af09b99053`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:25,846::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:25,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`049ad723-3205-4077-9bec-57af09b99053`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:25,847::task::595::Storage.TaskManager.Task::(_updateState) Task=`049ad723-3205-4077-9bec-57af09b99053`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:25,847::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:25,847::task::993::Storage.TaskManager.Task::(_decref) Task=`049ad723-3205-4077-9bec-57af09b99053`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:48:26,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b72cbb4-0411-4321-b155-0886ee5759e8`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:26,433::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:48:26,433::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000331995', 'lastCheck': '9.9', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000303805', 'lastCheck': '9.4', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:26,433::task::1191::Storage.TaskManager.Task::(prepare) Task=`1b72cbb4-0411-4321-b155-0886ee5759e8`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000331995', 'lastCheck': '9.9', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000303805', 'lastCheck': '9.4', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:26,433::task::595::Storage.TaskManager.Task::(_updateState) Task=`1b72cbb4-0411-4321-b155-0886ee5759e8`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:26,433::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:26,433::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:26,434::task::993::Storage.TaskManager.Task::(_decref) Task=`1b72cbb4-0411-4321-b155-0886ee5759e8`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:26,581::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:26,581::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:26,581::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:26,581::__init__::232::IOProcess::(_processLogs) (50537) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:26,581::__init__::232::IOProcess::(_processLogs) (50537) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:26,582::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:26,588::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000359002 s, 1.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:26,588::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:26,589::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:26,589::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:26,589::__init__::232::IOProcess::(_processLogs) (50538) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:26,589::__init__::232::IOProcess::(_processLogs) (50538) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:27,037::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:27,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:27,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:27,037::__init__::232::IOProcess::(_processLogs) (50539) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:27,038::__init__::232::IOProcess::(_processLogs) (50539) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:27,038::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:27,044::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000299547 s, 1.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:27,045::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:27,045::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:27,045::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:27,045::__init__::232::IOProcess::(_processLogs) (50540) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:27,045::__init__::232::IOProcess::(_processLogs) (50540) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:27,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:27,046::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:27,046::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:27,046::__init__::232::IOProcess::(_processLogs) (50541) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:27,046::__init__::232::IOProcess::(_processLogs) (50541) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) (50542) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) (50542) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:27,047::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:27,048::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:27,048::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:27,048::__init__::232::IOProcess::(_processLogs) (50543) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:27,048::__init__::232::IOProcess::(_processLogs) (50543) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:27,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:27,420::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115971 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:29,429::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:29,470::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116269 s, 88.1 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:31,479::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:31,519::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118405 s, 86.5 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:33,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:33,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116663 s, 87.8 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:35,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:35,625::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0207931 s, 49.2 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:35,897::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:35,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b30fe57-2adf-4e3b-b26b-ad891c4f0fd6`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:35,897::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:35,922::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:35,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`8b30fe57-2adf-4e3b-b26b-ad891c4f0fd6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:35,922::task::595::Storage.TaskManager.Task::(_updateState) Task=`8b30fe57-2adf-4e3b-b26b-ad891c4f0fd6`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:35,922::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:35,922::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:35,922::task::993::Storage.TaskManager.Task::(_decref) Task=`8b30fe57-2adf-4e3b-b26b-ad891c4f0fd6`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:35,933::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:35,933::task::595::Storage.TaskManager.Task::(_updateState) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:35,933::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:35,934::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`6e83b705-a4c0-4efb-8280-63d57b54978c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:35,934::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:35,934::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:35,934::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`6e83b705-a4c0-4efb-8280-63d57b54978c`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:35,934::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:35,934::task::993::Storage.TaskManager.Task::(_decref) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:35,953::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:35,953::task::1191::Storage.TaskManager.Task::(prepare) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:35,953::task::595::Storage.TaskManager.Task::(_updateState) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:35,953::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:35,953::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:35,954::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:35,954::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:35,954::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:35,954::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:35,954::task::993::Storage.TaskManager.Task::(_decref) Task=`f74bcd5a-f1ef-4df0-a981-64b52e9d961c`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:36,590::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:36,590::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:36,591::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:36,591::__init__::232::IOProcess::(_processLogs) (50544) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:36,591::__init__::232::IOProcess::(_processLogs) (50544) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:36,591::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:36,597::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000377124 s, 1.2 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:36,598::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:36,598::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:36,598::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:36,598::__init__::232::IOProcess::(_processLogs) (50545) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:36,599::__init__::232::IOProcess::(_processLogs) (50545) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:37,049::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:37,049::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:37,049::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:37,049::__init__::232::IOProcess::(_processLogs) (50546) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:37,049::__init__::232::IOProcess::(_processLogs) (50546) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:37,050::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:37,056::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000414444 s, 775 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:37,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:37,057::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:37,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:37,057::__init__::232::IOProcess::(_processLogs) (50547) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:37,057::__init__::232::IOProcess::(_processLogs) (50547) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) (50548) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) (50548) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:37,058::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) (50549) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) (50549) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:37,059::__init__::232::IOProcess::(_processLogs) (50550) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:37,060::__init__::232::IOProcess::(_processLogs) (50550) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:37,634::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:37,681::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0187351 s, 54.7 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:39,690::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:39,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191889 s, 53.4 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:41,624::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d345c27-53db-4913-91a8-26f0b0d8ed23`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:41,624::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:48:41,625::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377124', 'lastCheck': '5.0', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414444', 'lastCheck': '4.6', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:41,625::task::1191::Storage.TaskManager.Task::(prepare) Task=`6d345c27-53db-4913-91a8-26f0b0d8ed23`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000377124', 'lastCheck': '5.0', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000414444', 'lastCheck': '4.6', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:41,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`6d345c27-53db-4913-91a8-26f0b0d8ed23`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:41,626::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:41,626::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:41,626::task::993::Storage.TaskManager.Task::(_decref) Task=`6d345c27-53db-4913-91a8-26f0b0d8ed23`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:48:41,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:41,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115192 s, 88.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:43,493::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [381ef64a] >Thread-49099::DEBUG::2014-10-23 16:48:43,494::task::595::Storage.TaskManager.Task::(_updateState) Task=`db8af567-1645-4a5b-92a6-b61efc8fcae2`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:43,494::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:48:43,494::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:43,512::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:43,513::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:43,526::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:43,526::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:43,536::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:43,536::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:43,537::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:43,666::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:43,667::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:43,667::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', '2d4b6e8f-878f-4d9e-96f5-176b679b43e4') >Thread-49099::DEBUG::2014-10-23 16:48:43,667::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49099::INFO::2014-10-23 16:48:43,667::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:48:43,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`db8af567-1645-4a5b-92a6-b61efc8fcae2`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:48:43,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`db8af567-1645-4a5b-92a6-b61efc8fcae2`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:43,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:43,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:43,668::task::993::Storage.TaskManager.Task::(_decref) Task=`db8af567-1645-4a5b-92a6-b61efc8fcae2`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:48:43,683::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [60b7c3be] >Thread-49099::DEBUG::2014-10-23 16:48:43,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`1522beca-126a-4353-bc3e-3012fe07edb4`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:43,683::logUtils::44::dispatcher::(wrapper) Run and protect: getVGInfo(vgUUID='bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', options=None) >Thread-49099::DEBUG::2014-10-23 16:48:43,710::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:43,710::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size /dev/mapper/1cmg02 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:43,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:43,837::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120943 s, 84.7 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:44,027::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:44,028::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::INFO::2014-10-23 16:48:44,028::logUtils::47::dispatcher::(wrapper) Run and protect: getVGInfo, Return response: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', 'vgfree': '97442070528', 'vgUUID': 'bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-49099::DEBUG::2014-10-23 16:48:44,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`1522beca-126a-4353-bc3e-3012fe07edb4`::finished: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', 'vgfree': '97442070528', 'vgUUID': 'bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-49099::DEBUG::2014-10-23 16:48:44,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`1522beca-126a-4353-bc3e-3012fe07edb4`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:44,028::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:44,028::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:44,029::task::993::Storage.TaskManager.Task::(_decref) Task=`1522beca-126a-4353-bc3e-3012fe07edb4`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:44,054::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [60b7c3be] >Thread-49098::DEBUG::2014-10-23 16:48:44,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:44,054::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID='2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:44,055::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`860a6816-a2fa-4f82-ac65-24b10ee6f7c9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1249' at 'activateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:48:44,055::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:48:44,055::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:44,055::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`860a6816-a2fa-4f82-ac65-24b10ee6f7c9`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:44,055::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:48:44,055::task::993::Storage.TaskManager.Task::(_decref) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:44,056::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`15c35723-f79b-4854-8665-abf7a054a0ea`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1250' at 'activateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:48:44,056::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:48:44,056::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:44,056::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`15c35723-f79b-4854-8665-abf7a054a0ea`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:44,056::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::_resourcesAcquired: Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (exclusive) >Thread-49098::DEBUG::2014-10-23 16:48:44,056::task::993::Storage.TaskManager.Task::(_decref) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:44,056::sp::1031::Storage.StoragePool::(activateSD) sdUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49098::DEBUG::2014-10-23 16:48:44,056::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49098::DEBUG::2014-10-23 16:48:44,056::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:48:44,056::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49098::DEBUG::2014-10-23 16:48:44,057::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:48:44,057::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49098::DEBUG::2014-10-23 16:48:44,057::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,071::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:48:44,072::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,182::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,182::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,182::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,183::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,183::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,183::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,183::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,183::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:48:44,183::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,185::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,296::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,297::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,297::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:48:44,297::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata count=1 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,303::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000577651 s, 3.5 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,304::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000577651 s, 3.5 MB/s'], size: 2048 >Thread-49098::DEBUG::2014-10-23 16:48:44,304::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-49098::DEBUG::2014-10-23 16:48:44,304::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-49098::DEBUG::2014-10-23 16:48:44,304::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:48:44,304::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,304::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,304::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,305::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,305::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,305::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,401::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,402::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,402::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', '_SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe'] >Thread-49098::INFO::2014-10-23 16:48:44,402::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_imageNS already registered >Thread-49098::INFO::2014-10-23 16:48:44,403::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_volumeNS already registered >Thread-49098::INFO::2014-10-23 16:48:44,403::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_lvmActivationNS already registered >Thread-49098::DEBUG::2014-10-23 16:48:44,403::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/images >Thread-49098::WARNING::2014-10-23 16:48:44,403::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/images already exists >Thread-49098::DEBUG::2014-10-23 16:48:44,403::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/dom_md >Thread-49098::WARNING::2014-10-23 16:48:44,403::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/dom_md already exists >Thread-49098::INFO::2014-10-23 16:48:44,403::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49098::DEBUG::2014-10-23 16:48:44,404::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49098::DEBUG::2014-10-23 16:48:44,404::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-49098::DEBUG::2014-10-23 16:48:44,404::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-49098::DEBUG::2014-10-23 16:48:44,404::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:48:44,404::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata count=1 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,410::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000391497 s, 5.2 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,410::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000391497 s, 5.2 MB/s'], size: 2048 >Thread-49098::DEBUG::2014-10-23 16:48:44,410::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-49098::DEBUG::2014-10-23 16:48:44,411::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-49098::DEBUG::2014-10-23 16:48:44,411::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:48:44,411::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,411::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,411::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,411::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,411::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,412::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,506::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:44,507::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:48:44,507::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', '_SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe'] >Thread-49098::INFO::2014-10-23 16:48:44,507::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_imageNS already registered >Thread-49098::INFO::2014-10-23 16:48:44,508::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_volumeNS already registered >Thread-49098::INFO::2014-10-23 16:48:44,508::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_lvmActivationNS already registered >Thread-49098::INFO::2014-10-23 16:48:44,508::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'Active', '748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49098::DEBUG::2014-10-23 16:48:44,508::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49098::DEBUG::2014-10-23 16:48:44,508::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::INFO::2014-10-23 16:48:44,508::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49098::DEBUG::2014-10-23 16:48:44,509::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `11cfd6be-5798-4973-9e8b-a3262a2d3ebe` started monitoring domain `2027a9ce-c630-4748-bb7c-1e8cc9c88ca2` >Thread-49247::DEBUG::2014-10-23 16:48:44,509::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49098::DEBUG::2014-10-23 16:48:44,509::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::INFO::2014-10-23 16:48:44,509::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:48:44,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::finished: None >Thread-49247::DEBUG::2014-10-23 16:48:44,510::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:48:44,511::task::595::Storage.TaskManager.Task::(_updateState) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:44,516::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': < ResourceRef 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:44,517::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:44,518::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:44,518::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:44,518::task::993::Storage.TaskManager.Task::(_decref) Task=`5a083a60-64a1-448f-94f1-7ccc5702efb0`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:48:44,556::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:44,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`4346ae11-c18b-41b8-8278-867d26264c47`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:44,557::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'}, options=None) >Thread-49099::DEBUG::2014-10-23 16:48:44,557::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`2ea3dd69-04c4-4b31-a2a8-2d3f79ebf338`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:48:44,557::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49099::DEBUG::2014-10-23 16:48:44,557::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:48:44,557::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`2ea3dd69-04c4-4b31-a2a8-2d3f79ebf338`::Granted request >Thread-49099::DEBUG::2014-10-23 16:48:44,558::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`e9a91d86-d88e-49cf-8fbf-cc9788e69f84`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:48:44,558::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49099::DEBUG::2014-10-23 16:48:44,558::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:48:44,558::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`e9a91d86-d88e-49cf-8fbf-cc9788e69f84`::Granted request >Thread-49099::DEBUG::2014-10-23 16:48:44,558::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'} >Thread-49099::INFO::2014-10-23 16:48:44,558::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'Active', '748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49099::DEBUG::2014-10-23 16:48:44,558::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,559::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:48:44,559::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:44,559::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:48:44,559::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:44,560::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:48:44,560::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:44,580::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:48:44,581::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49247::DEBUG::2014-10-23 16:48:44,603::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:44,603::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:48:44,651::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:44,651::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,651::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,651::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,651::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,652::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,652::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,652::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:44,652::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49247::DEBUG::2014-10-23 16:48:44,652::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49247::DEBUG::2014-10-23 16:48:44,653::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49247::DEBUG::2014-10-23 16:48:44,655::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-41::DEBUG::2014-10-23 16:48:44,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-49247::DEBUG::2014-10-23 16:48:44,657::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-41::DEBUG::2014-10-23 16:48:44,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,664::__init__::232::IOProcess::(_processLogs) (50551) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:44,664::__init__::232::IOProcess::(_processLogs) (50551) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:44,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:44,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,668::__init__::232::IOProcess::(_processLogs) (50552) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:44,668::__init__::232::IOProcess::(_processLogs) (50552) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:44,669::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49247::DEBUG::2014-10-23 16:48:44,671::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:48:44,672::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:44,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,672::__init__::232::IOProcess::(_processLogs) (50553) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:48:44,672::__init__::232::IOProcess::(_processLogs) (50553) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:44,673::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:48:44,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:44,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,674::__init__::232::IOProcess::(_processLogs) (50554) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:48:44,674::__init__::232::IOProcess::(_processLogs) (50554) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:44,674::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:48:44,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:44,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,676::__init__::232::IOProcess::(_processLogs) (50555) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:44,676::__init__::232::IOProcess::(_processLogs) (50555) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:44,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:44,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:44,677::__init__::232::IOProcess::(_processLogs) (50556) Got request for method 'glob' >Thread-49247::DEBUG::2014-10-23 16:48:44,678::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-41::DEBUG::2014-10-23 16:48:44,683::__init__::232::IOProcess::(_processLogs) (50556) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:44,684::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49099::INFO::2014-10-23 16:48:44,684::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49099::INFO::2014-10-23 16:48:44,684::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49099::DEBUG::2014-10-23 16:48:44,684::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-49099::DEBUG::2014-10-23 16:48:44,685::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49099::DEBUG::2014-10-23 16:48:44,685::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:44,685::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:48:44,685::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49099::WARNING::2014-10-23 16:48:44,685::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe already exists >Thread-49099::DEBUG::2014-10-23 16:48:44,685::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,686::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 748c2d80-0413-4285-ae64-fade24166b4b (cwd None) >Thread-49247::DEBUG::2014-10-23 16:48:44,784::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:44,785::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Thread-49099::DEBUG::2014-10-23 16:48:44,791::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found\n Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b\n'; <rc> = 5 >Thread-49099::WARNING::2014-10-23 16:48:44,792::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [' bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL|2027a9ce-c630-4748-bb7c-1e8cc9c88ca2|wz--n-|101602820096|97442070528|134217728|757|726|MDT_LEASETIMESEC=60,MDT_ROLE=Regular,MDT_CLASS=Data,MDT_VERSION=3,MDT_TYPE=ISCSI,MDT_LOGBLKSIZE=512,MDT_LEASERETRIES=3,MDT_IOOPTIMEOUTSEC=10,MDT_LOCKRENEWALINTERVALSEC=5,MDT_LOCKPOLICY=,MDT_VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL,MDT_PHYBLKSIZE=512,MDT_DESCRIPTION=iscsi_0,MDT_SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2,MDT_PV0=pv:1cmg02&44&uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ&44&pestart:0&44&pecount:757&44&mapoffset:0,RHAT_storage_domain,MDT__SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e,MDT_POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe|134217728|67104768|6|1|/dev/mapper/1cmg02'] [' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found', ' Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b'] >Thread-49099::DEBUG::2014-10-23 16:48:44,792::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:44,793::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/dom_md >Thread-49247::DEBUG::2014-10-23 16:48:44,793::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::WARNING::2014-10-23 16:48:44,793::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/dom_md already exists >Thread-49099::DEBUG::2014-10-23 16:48:44,794::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/images >Thread-49247::DEBUG::2014-10-23 16:48:44,794::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:48:44,794::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/images already exists >Thread-49247::DEBUG::2014-10-23 16:48:44,795::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,797::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:48:44,798::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-49247::DEBUG::2014-10-23 16:48:44,798::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-41::DEBUG::2014-10-23 16:48:44,798::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-49247::DEBUG::2014-10-23 16:48:44,799::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:48:44,799::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-49247::DEBUG::2014-10-23 16:48:44,800::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:48:44,800::__init__::232::IOProcess::(_processLogs) (50557) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:44,801::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-49247::DEBUG::2014-10-23 16:48:44,801::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-41::DEBUG::2014-10-23 16:48:44,801::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:44,802::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-49247::DEBUG::2014-10-23 16:48:44,802::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-41::DEBUG::2014-10-23 16:48:44,802::__init__::232::IOProcess::(_processLogs) (50558) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:44,808::__init__::232::IOProcess::(_processLogs) (50557) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:44,808::__init__::232::IOProcess::(_processLogs) (50558) Queuing response >Thread-49099::INFO::2014-10-23 16:48:44,811::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49099::DEBUG::2014-10-23 16:48:44,812::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49099::INFO::2014-10-23 16:48:44,812::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::DEBUG::2014-10-23 16:48:44,812::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::INFO::2014-10-23 16:48:44,813::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::DEBUG::2014-10-23 16:48:44,813::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::INFO::2014-10-23 16:48:44,813::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:48:44,813::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:48:44,813::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49099::DEBUG::2014-10-23 16:48:44,813::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:48:44,813::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:48:44,813::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49099::INFO::2014-10-23 16:48:44,814::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49099::DEBUG::2014-10-23 16:48:44,814::task::1191::Storage.TaskManager.Task::(prepare) Task=`4346ae11-c18b-41b8-8278-867d26264c47`::finished: True >Thread-49099::DEBUG::2014-10-23 16:48:44,814::task::595::Storage.TaskManager.Task::(_updateState) Task=`4346ae11-c18b-41b8-8278-867d26264c47`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:44,814::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:44,814::task::993::Storage.TaskManager.Task::(_decref) Task=`4346ae11-c18b-41b8-8278-867d26264c47`::ref 0 aborting False >Thread-49247::DEBUG::2014-10-23 16:48:44,897::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:44,897::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,897::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-49247::DEBUG::2014-10-23 16:48:44,898::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata count=1 (cwd None) >Thread-49247::DEBUG::2014-10-23 16:48:44,903::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000373925 s, 5.5 MB/s\n'; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:44,904::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000373925 s, 5.5 MB/s'], size: 2048 >Thread-49247::DEBUG::2014-10-23 16:48:44,904::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-49247::DEBUG::2014-10-23 16:48:44,904::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-49247::DEBUG::2014-10-23 16:48:44,904::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-49247::DEBUG::2014-10-23 16:48:44,904::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,904::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,904::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,905::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,905::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:44,905::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49247::DEBUG::2014-10-23 16:48:45,000::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:45,000::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49247::DEBUG::2014-10-23 16:48:45,000::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', '_SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe'] >Thread-49247::INFO::2014-10-23 16:48:45,001::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_imageNS already registered >Thread-49247::INFO::2014-10-23 16:48:45,001::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_volumeNS already registered >Thread-49247::INFO::2014-10-23 16:48:45,001::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_lvmActivationNS already registered >Thread-49247::DEBUG::2014-10-23 16:48:45,001::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49247::DEBUG::2014-10-23 16:48:45,007::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00051263 s, 8.0 MB/s\n'; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:45,008::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 changed its status to Valid >Thread-49252::DEBUG::2014-10-23 16:48:45,008::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-49247::INFO::2014-10-23 16:48:45,008::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (id: 1) >Thread-49252::DEBUG::2014-10-23 16:48:45,009::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-49247::DEBUG::2014-10-23 16:48:45,009::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 successfully acquired (id: 1) >Thread-49252::DEBUG::2014-10-23 16:48:45,010::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_upgradePoolDomain` >Thread-49252::DEBUG::2014-10-23 16:48:45,010::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-49253::INFO::2014-10-23 16:48:45,013::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Dummy-2584::DEBUG::2014-10-23 16:48:45,844::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:45,884::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115322 s, 88.8 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:46,179::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:46,179::task::595::Storage.TaskManager.Task::(_updateState) Task=`11c4a8d2-736b-41c4-8b79-70750252e3d4`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:46,180::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:46,191::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:46,191::task::1191::Storage.TaskManager.Task::(prepare) Task=`11c4a8d2-736b-41c4-8b79-70750252e3d4`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:46,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`11c4a8d2-736b-41c4-8b79-70750252e3d4`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:46,191::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:46,191::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:46,191::task::993::Storage.TaskManager.Task::(_decref) Task=`11c4a8d2-736b-41c4-8b79-70750252e3d4`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:46,202::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:46,202::task::595::Storage.TaskManager.Task::(_updateState) Task=`d92ea334-926b-421c-84fe-698334150ee9`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:46,202::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:46,203::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d0a66392-1f91-41f2-a603-046a68209406`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:46,203::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:46,203::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:46,203::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d0a66392-1f91-41f2-a603-046a68209406`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:46,203::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d92ea334-926b-421c-84fe-698334150ee9`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:46,203::task::993::Storage.TaskManager.Task::(_decref) Task=`d92ea334-926b-421c-84fe-698334150ee9`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:46,216::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Active,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Active', 'diskfree': '97442070528', 'isoprefix': '', 'alerts': [], 'disktotal': '101602820096', 'version': 3}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:46,216::task::1191::Storage.TaskManager.Task::(prepare) Task=`d92ea334-926b-421c-84fe-698334150ee9`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Active,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Active', 'diskfree': '97442070528', 'isoprefix': '', 'alerts': [], 'disktotal': '101602820096', 'version': 3}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:46,216::task::595::Storage.TaskManager.Task::(_updateState) Task=`d92ea334-926b-421c-84fe-698334150ee9`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:46,216::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:46,217::task::993::Storage.TaskManager.Task::(_decref) Task=`d92ea334-926b-421c-84fe-698334150ee9`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:46,601::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,601::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,601::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,601::__init__::232::IOProcess::(_processLogs) (50559) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:46,601::__init__::232::IOProcess::(_processLogs) (50559) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:46,602::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,602::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,602::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,602::__init__::232::IOProcess::(_processLogs) (50560) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:46,602::__init__::232::IOProcess::(_processLogs) (50560) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:46,604::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:48:46,604::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,604::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,604::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,604::__init__::232::IOProcess::(_processLogs) (50561) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:48:46,605::__init__::232::IOProcess::(_processLogs) (50561) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:46,605::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:48:46,605::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,606::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,606::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,606::__init__::232::IOProcess::(_processLogs) (50562) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:48:46,606::__init__::232::IOProcess::(_processLogs) (50562) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:46,606::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-41::DEBUG::2014-10-23 16:48:46,607::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,607::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,607::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,607::__init__::232::IOProcess::(_processLogs) (50563) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:46,607::__init__::232::IOProcess::(_processLogs) (50563) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:46,608::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,608::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,608::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,608::__init__::232::IOProcess::(_processLogs) (50564) Got request for method 'glob' >Thread-49139::DEBUG::2014-10-23 16:48:46,609::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-41::DEBUG::2014-10-23 16:48:46,609::__init__::232::IOProcess::(_processLogs) (50564) Queuing response >Thread-49139::INFO::2014-10-23 16:48:46,609::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49139::INFO::2014-10-23 16:48:46,609::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:48:46,610::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,610::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,610::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,610::__init__::232::IOProcess::(_processLogs) (50565) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:46,610::__init__::232::IOProcess::(_processLogs) (50565) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:46,611::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:46,617::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000313494 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:46,618::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:46,618::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:46,618::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:46,618::__init__::232::IOProcess::(_processLogs) (50566) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:46,618::__init__::232::IOProcess::(_processLogs) (50566) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:47,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:47,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:47,061::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:47,061::__init__::232::IOProcess::(_processLogs) (50567) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:47,061::__init__::232::IOProcess::(_processLogs) (50567) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:47,062::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:47,068::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000295955 s, 1.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:47,069::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:47,069::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:47,069::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:47,069::__init__::232::IOProcess::(_processLogs) (50568) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:47,069::__init__::232::IOProcess::(_processLogs) (50568) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:47,070::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:47,070::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:47,070::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:47,070::__init__::232::IOProcess::(_processLogs) (50569) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:47,070::__init__::232::IOProcess::(_processLogs) (50569) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:47,071::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:47,071::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:47,071::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:47,071::__init__::232::IOProcess::(_processLogs) (50570) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:47,071::__init__::232::IOProcess::(_processLogs) (50570) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:47,072::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:47,072::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:47,072::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:47,072::__init__::232::IOProcess::(_processLogs) (50571) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:47,072::__init__::232::IOProcess::(_processLogs) (50571) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:47,893::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:47,946::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0229149 s, 44.7 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:48:49,192::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [2ccea5b2] >Thread-49098::DEBUG::2014-10-23 16:48:49,192::task::595::Storage.TaskManager.Task::(_updateState) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:49,192::logUtils::44::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID='2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=2, options=None) >Thread-49098::DEBUG::2014-10-23 16:48:49,193::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`46fa0150-ead7-4363-9f9e-fb8e7fda1472`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1227' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:48:49,193::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:48:49,193::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:49,193::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`46fa0150-ead7-4363-9f9e-fb8e7fda1472`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:49,193::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:48:49,193::task::993::Storage.TaskManager.Task::(_decref) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:49,194::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`95e6da18-7bb2-43b9-aa10-adff7733be4d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1228' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:48:49,194::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:48:49,194::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:49,194::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`95e6da18-7bb2-43b9-aa10-adff7733be4d`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:49,194::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::_resourcesAcquired: Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (exclusive) >Thread-49098::DEBUG::2014-10-23 16:48:49,194::task::993::Storage.TaskManager.Task::(_decref) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:49,194::sp::1070::Storage.StoragePool::(deactivateSD) sdUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-49098::INFO::2014-10-23 16:48:49,195::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'Attached', '748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49098::DEBUG::2014-10-23 16:48:49,195::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49098::DEBUG::2014-10-23 16:48:49,196::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::INFO::2014-10-23 16:48:49,196::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49247::DEBUG::2014-10-23 16:48:49,196::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49247::INFO::2014-10-23 16:48:49,196::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (id: 1) >Dummy-2584::DEBUG::2014-10-23 16:48:50,071::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:50,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124478 s, 82.3 MB/s\n'; <rc> = 0 >Thread-49247::DEBUG::2014-10-23 16:48:51,196::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 released successfully (id: 1) >Thread-49098::DEBUG::2014-10-23 16:48:51,197::sp::1407::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `11cfd6be-5798-4973-9e8b-a3262a2d3ebe` stopped monitoring domain `2027a9ce-c630-4748-bb7c-1e8cc9c88ca2` >Thread-49098::DEBUG::2014-10-23 16:48:51,197::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::INFO::2014-10-23 16:48:51,197::logUtils::47::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:48:51,197::task::1191::Storage.TaskManager.Task::(prepare) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::finished: None >Thread-49098::DEBUG::2014-10-23 16:48:51,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:51,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': < ResourceRef 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:51,198::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:51,199::task::993::Storage.TaskManager.Task::(_decref) Task=`76c72211-2bd0-4f20-a76e-04548831bcb1`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:48:51,211::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:51,211::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6153ae1-d276-4ac8-bb34-f6e5caf941be`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:51,211::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'attached', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'}, options=None) >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`5aca1eda-3036-463e-a20e-012118b7b872`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`5aca1eda-3036-463e-a20e-012118b7b872`::Granted request >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`e438cc55-d289-404c-96bf-51aeef1387da`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49099::DEBUG::2014-10-23 16:48:51,212::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:48:51,213::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`e438cc55-d289-404c-96bf-51aeef1387da`::Granted request >Thread-49099::DEBUG::2014-10-23 16:48:51,213::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'attached', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'} >Thread-49099::INFO::2014-10-23 16:48:51,213::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': 'Attached', '748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49099::DEBUG::2014-10-23 16:48:51,213::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,213::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,213::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,213::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,213::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,214::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,214::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:48:51,214::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:51,214::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:48:51,214::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:51,214::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:48:51,214::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:51,229::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:48:51,235::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:51,359::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,360::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:48:51,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,362::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,362::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) (50572) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) (50573) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) (50572) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:51,363::__init__::232::IOProcess::(_processLogs) (50573) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:51,365::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:48:51,366::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,366::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,366::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,366::__init__::232::IOProcess::(_processLogs) (50574) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:48:51,367::__init__::232::IOProcess::(_processLogs) (50574) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:51,367::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:48:51,367::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,368::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,368::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,368::__init__::232::IOProcess::(_processLogs) (50575) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:48:51,368::__init__::232::IOProcess::(_processLogs) (50575) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:51,368::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:48:51,369::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,369::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,369::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,369::__init__::232::IOProcess::(_processLogs) (50576) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) (50576) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) (50577) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:51,370::__init__::232::IOProcess::(_processLogs) (50577) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:51,370::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49099::INFO::2014-10-23 16:48:51,370::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49099::INFO::2014-10-23 16:48:51,371::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49099::DEBUG::2014-10-23 16:48:51,371::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-49099::DEBUG::2014-10-23 16:48:51,371::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49099::DEBUG::2014-10-23 16:48:51,371::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:51,371::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:48:51,371::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49099::WARNING::2014-10-23 16:48:51,371::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe already exists >Thread-49099::DEBUG::2014-10-23 16:48:51,371::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:51,372::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 748c2d80-0413-4285-ae64-fade24166b4b (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:51,471::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found\n Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b\n'; <rc> = 5 >Thread-49099::WARNING::2014-10-23 16:48:51,472::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found', ' Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b'] >Thread-49099::DEBUG::2014-10-23 16:48:51,472::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:48:51,474::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) (50578) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) (50579) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) (50578) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:51,475::__init__::232::IOProcess::(_processLogs) (50579) Queuing response >Thread-49099::INFO::2014-10-23 16:48:51,476::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::DEBUG::2014-10-23 16:48:51,477::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::INFO::2014-10-23 16:48:51,477::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::DEBUG::2014-10-23 16:48:51,477::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::INFO::2014-10-23 16:48:51,477::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:48:51,477::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:48:51,477::sp::1221::Storage.StoragePool::(__rebuild) collecting stale storage domain link /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49099::DEBUG::2014-10-23 16:48:51,477::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49099::DEBUG::2014-10-23 16:48:51,477::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:48:51,478::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49099::INFO::2014-10-23 16:48:51,478::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49099::DEBUG::2014-10-23 16:48:51,478::task::1191::Storage.TaskManager.Task::(prepare) Task=`e6153ae1-d276-4ac8-bb34-f6e5caf941be`::finished: True >Thread-49099::DEBUG::2014-10-23 16:48:51,478::task::595::Storage.TaskManager.Task::(_updateState) Task=`e6153ae1-d276-4ac8-bb34-f6e5caf941be`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:51,479::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:51,479::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:51,479::task::993::Storage.TaskManager.Task::(_decref) Task=`e6153ae1-d276-4ac8-bb34-f6e5caf941be`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:48:51,527::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [2ccea5b2] >Thread-49099::DEBUG::2014-10-23 16:48:51,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8185ce9-8090-4597-9473-e78f18774a2f`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:51,528::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:48:51,530::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -r 1 -u (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:52,063::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:52,063::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:48:52,063::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:52,063::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:48:52,063::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:48:52,063::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:48:52,064::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:48:52,074::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:48:52,075::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:52,121::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:52,162::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116175 s, 88.1 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:52,169::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:52,170::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,171::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,172::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,173::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,173::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,174::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:48:52,175::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::INFO::2014-10-23 16:48:52,175::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:48:52,176::task::1191::Storage.TaskManager.Task::(prepare) Task=`b8185ce9-8090-4597-9473-e78f18774a2f`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:48:52,177::task::595::Storage.TaskManager.Task::(_updateState) Task=`b8185ce9-8090-4597-9473-e78f18774a2f`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:52,177::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:52,177::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:52,178::task::993::Storage.TaskManager.Task::(_decref) Task=`b8185ce9-8090-4597-9473-e78f18774a2f`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:48:54,183::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:54,224::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116948 s, 87.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:48:56,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:56,276::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116445 s, 87.9 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:48:56,277::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:48:56,279::task::595::Storage.TaskManager.Task::(_updateState) Task=`a037f2c4-df80-4bbb-8b38-b20b9a9e0c85`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:56,279::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:48:56,291::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:56,291::task::1191::Storage.TaskManager.Task::(prepare) Task=`a037f2c4-df80-4bbb-8b38-b20b9a9e0c85`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:48:56,291::task::595::Storage.TaskManager.Task::(_updateState) Task=`a037f2c4-df80-4bbb-8b38-b20b9a9e0c85`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:56,291::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:56,291::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:56,291::task::993::Storage.TaskManager.Task::(_decref) Task=`a037f2c4-df80-4bbb-8b38-b20b9a9e0c85`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:48:56,303::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:48:56,303::task::595::Storage.TaskManager.Task::(_updateState) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:48:56,303::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:48:56,303::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`940b878a-237a-4149-bd65-0755db8c99bd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:48:56,304::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:48:56,304::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:48:56,304::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`940b878a-237a-4149-bd65-0755db8c99bd`::Granted request >Thread-49098::DEBUG::2014-10-23 16:48:56,305::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:48:56,305::task::993::Storage.TaskManager.Task::(_decref) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:48:56,317::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:56,317::task::1191::Storage.TaskManager.Task::(prepare) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:48:56,317::task::595::Storage.TaskManager.Task::(_updateState) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:48:56,317::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:48:56,318::task::993::Storage.TaskManager.Task::(_decref) Task=`f521f20e-350d-4515-a9fe-ddd6c78f10dd`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:56,620::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) (50580) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) (50581) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:48:56,621::__init__::232::IOProcess::(_processLogs) (50580) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:56,622::__init__::232::IOProcess::(_processLogs) (50581) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:56,623::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:48:56,624::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,624::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,624::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,624::__init__::232::IOProcess::(_processLogs) (50582) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:48:56,625::__init__::232::IOProcess::(_processLogs) (50582) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:56,625::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:48:56,625::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,626::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,626::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,626::__init__::232::IOProcess::(_processLogs) (50583) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:48:56,626::__init__::232::IOProcess::(_processLogs) (50583) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:56,627::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-41::DEBUG::2014-10-23 16:48:56,627::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,627::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,627::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,627::__init__::232::IOProcess::(_processLogs) (50584) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:56,627::__init__::232::IOProcess::(_processLogs) (50584) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:56,628::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,628::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,628::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,628::__init__::232::IOProcess::(_processLogs) (50585) Got request for method 'glob' >Thread-49139::DEBUG::2014-10-23 16:48:56,629::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-41::DEBUG::2014-10-23 16:48:56,629::__init__::232::IOProcess::(_processLogs) (50585) Queuing response >Thread-49139::INFO::2014-10-23 16:48:56,629::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49139::INFO::2014-10-23 16:48:56,629::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:48:56,630::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,630::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,630::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,630::__init__::232::IOProcess::(_processLogs) (50586) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:56,631::__init__::232::IOProcess::(_processLogs) (50586) Queuing response >Thread-49139::DEBUG::2014-10-23 16:48:56,631::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:48:56,637::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000321228 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:56,638::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:56,638::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:56,638::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:56,638::__init__::232::IOProcess::(_processLogs) (50587) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:56,638::__init__::232::IOProcess::(_processLogs) (50587) Queuing response >Thread-49099::DEBUG::2014-10-23 16:48:56,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`86287bff-3c2f-4705-8b0d-a37157ced6e7`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:48:56,880::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:48:56,880::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000321228', 'lastCheck': '0.2', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295955', 'lastCheck': '9.8', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:56,880::task::1191::Storage.TaskManager.Task::(prepare) Task=`86287bff-3c2f-4705-8b0d-a37157ced6e7`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000321228', 'lastCheck': '0.2', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295955', 'lastCheck': '9.8', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:48:56,880::task::595::Storage.TaskManager.Task::(_updateState) Task=`86287bff-3c2f-4705-8b0d-a37157ced6e7`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:48:56,880::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:48:56,880::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:48:56,880::task::993::Storage.TaskManager.Task::(_decref) Task=`86287bff-3c2f-4705-8b0d-a37157ced6e7`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:48:57,073::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:57,073::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:57,073::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:57,074::__init__::232::IOProcess::(_processLogs) (50588) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:57,074::__init__::232::IOProcess::(_processLogs) (50588) Queuing response >Thread-49080::DEBUG::2014-10-23 16:48:57,074::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:48:57,080::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000340222 s, 944 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:48:57,081::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:57,081::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:57,081::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:57,081::__init__::232::IOProcess::(_processLogs) (50589) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:48:57,081::__init__::232::IOProcess::(_processLogs) (50589) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:57,082::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:57,082::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:57,082::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:57,082::__init__::232::IOProcess::(_processLogs) (50590) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:57,082::__init__::232::IOProcess::(_processLogs) (50590) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:57,083::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:57,083::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:57,083::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:57,083::__init__::232::IOProcess::(_processLogs) (50591) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:57,083::__init__::232::IOProcess::(_processLogs) (50591) Queuing response >Thread-41::DEBUG::2014-10-23 16:48:57,084::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:48:57,084::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:48:57,084::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:48:57,084::__init__::232::IOProcess::(_processLogs) (50592) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:48:57,084::__init__::232::IOProcess::(_processLogs) (50592) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:48:58,289::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:48:58,331::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131476 s, 77.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:00,340::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:00,382::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0136872 s, 74.8 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:02,391::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:02,435::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115121 s, 88.9 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:04,444::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:04,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117028 s, 87.5 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:06,362::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:49:06,362::task::595::Storage.TaskManager.Task::(_updateState) Task=`e649313a-8527-48c6-ba33-87f73eee70ab`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:06,362::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:49:06,380::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:06,380::task::1191::Storage.TaskManager.Task::(prepare) Task=`e649313a-8527-48c6-ba33-87f73eee70ab`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:06,380::task::595::Storage.TaskManager.Task::(_updateState) Task=`e649313a-8527-48c6-ba33-87f73eee70ab`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:06,380::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:06,380::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:06,380::task::993::Storage.TaskManager.Task::(_decref) Task=`e649313a-8527-48c6-ba33-87f73eee70ab`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:06,397::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:06,397::task::595::Storage.TaskManager.Task::(_updateState) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:06,397::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:49:06,398::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`91cecadc-4c6d-4f53-b27e-2ceac03b8fa3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:06,398::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:06,398::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:06,398::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`91cecadc-4c6d-4f53-b27e-2ceac03b8fa3`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:06,398::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:06,398::task::993::Storage.TaskManager.Task::(_decref) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:06,411::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:06,411::task::1191::Storage.TaskManager.Task::(prepare) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:06,411::task::595::Storage.TaskManager.Task::(_updateState) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:06,411::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:06,411::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:06,411::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:06,411::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:06,412::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:06,412::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:06,412::task::993::Storage.TaskManager.Task::(_decref) Task=`7116ea95-f7d3-484b-baec-e37d51fab6c2`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:06,494::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:06,535::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117325 s, 87.3 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:06,640::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:06,640::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:06,640::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:06,640::__init__::232::IOProcess::(_processLogs) (50593) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:06,640::__init__::232::IOProcess::(_processLogs) (50593) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:06,641::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:49:06,647::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.00028389 s, 1.6 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:06,647::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:06,648::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:06,648::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:06,648::__init__::232::IOProcess::(_processLogs) (50594) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:06,648::__init__::232::IOProcess::(_processLogs) (50594) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:07,085::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:07,086::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:07,086::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:07,086::__init__::232::IOProcess::(_processLogs) (50595) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:07,086::__init__::232::IOProcess::(_processLogs) (50595) Queuing response >Thread-49080::DEBUG::2014-10-23 16:49:07,086::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:49:07,093::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000337819 s, 950 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:07,094::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:07,094::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:07,094::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:07,094::__init__::232::IOProcess::(_processLogs) (50596) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:07,094::__init__::232::IOProcess::(_processLogs) (50596) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:07,095::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:07,095::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:07,095::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:07,095::__init__::232::IOProcess::(_processLogs) (50597) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:07,095::__init__::232::IOProcess::(_processLogs) (50597) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) (50598) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) (50598) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:07,096::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:07,097::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:07,097::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:07,097::__init__::232::IOProcess::(_processLogs) (50599) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:07,097::__init__::232::IOProcess::(_processLogs) (50599) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:49:08,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:08,584::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116727 s, 87.7 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:10,593::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:10,633::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116331 s, 88.0 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:12,067::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b196ed1-f424-4b25-8393-850d39ab9823`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:12,067::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49099::INFO::2014-10-23 16:49:12,068::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00028389', 'lastCheck': '5.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000337819', 'lastCheck': '5.0', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:49:12,068::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b196ed1-f424-4b25-8393-850d39ab9823`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00028389', 'lastCheck': '5.4', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000337819', 'lastCheck': '5.0', 'valid': True}} >Thread-49099::DEBUG::2014-10-23 16:49:12,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b196ed1-f424-4b25-8393-850d39ab9823`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:12,068::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:12,068::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:12,068::task::993::Storage.TaskManager.Task::(_decref) Task=`6b196ed1-f424-4b25-8393-850d39ab9823`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:12,642::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:12,688::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0172408 s, 59.4 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:14,697::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:14,737::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116952 s, 87.6 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:16,457::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:49:16,458::task::595::Storage.TaskManager.Task::(_updateState) Task=`f87e7fb8-cebc-4cb0-9cfb-47cf6163eebf`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:16,458::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:49:16,469::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:16,469::task::1191::Storage.TaskManager.Task::(prepare) Task=`f87e7fb8-cebc-4cb0-9cfb-47cf6163eebf`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:16,469::task::595::Storage.TaskManager.Task::(_updateState) Task=`f87e7fb8-cebc-4cb0-9cfb-47cf6163eebf`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:16,469::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:16,469::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:16,469::task::993::Storage.TaskManager.Task::(_decref) Task=`f87e7fb8-cebc-4cb0-9cfb-47cf6163eebf`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:16,480::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:16,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:16,480::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:49:16,481::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`122ddb5e-59cc-4e50-bffe-399ead33babb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:16,481::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:16,481::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:16,481::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`122ddb5e-59cc-4e50-bffe-399ead33babb`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:16,481::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:16,481::task::993::Storage.TaskManager.Task::(_decref) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:16,498::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:16,498::task::1191::Storage.TaskManager.Task::(prepare) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '2027a9ce-c630-4748-bb7c-1e8cc9c88ca2:Attached,748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:16,498::task::595::Storage.TaskManager.Task::(_updateState) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:16,498::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:16,498::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:16,499::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:16,499::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:16,499::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:16,499::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:16,499::task::993::Storage.TaskManager.Task::(_decref) Task=`bf50cfe1-be89-4681-8e2d-8b8c8cea08c5`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:49:16,649::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:16,649::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:16,649::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:16,649::__init__::232::IOProcess::(_processLogs) (50600) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:16,650::__init__::232::IOProcess::(_processLogs) (50600) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:16,650::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:49:16,656::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000326203 s, 1.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:16,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:16,657::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:16,657::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:16,657::__init__::232::IOProcess::(_processLogs) (50601) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:16,657::__init__::232::IOProcess::(_processLogs) (50601) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:49:16,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:16,786::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118178 s, 86.6 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:17,098::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:17,098::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:17,098::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:17,098::__init__::232::IOProcess::(_processLogs) (50602) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:17,099::__init__::232::IOProcess::(_processLogs) (50602) Queuing response >Thread-49080::DEBUG::2014-10-23 16:49:17,099::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:49:17,105::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.00036526 s, 879 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:17,106::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:17,106::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:17,106::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:17,107::__init__::232::IOProcess::(_processLogs) (50603) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:17,107::__init__::232::IOProcess::(_processLogs) (50603) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:17,107::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:17,108::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:17,108::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:17,108::__init__::232::IOProcess::(_processLogs) (50604) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:17,108::__init__::232::IOProcess::(_processLogs) (50604) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:17,109::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:17,109::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:17,109::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:17,109::__init__::232::IOProcess::(_processLogs) (50605) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:17,109::__init__::232::IOProcess::(_processLogs) (50605) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:17,110::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:17,110::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:17,110::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:17,110::__init__::232::IOProcess::(_processLogs) (50606) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:17,110::__init__::232::IOProcess::(_processLogs) (50606) Queuing response >Dummy-2584::DEBUG::2014-10-23 16:49:18,795::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:18,835::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116388 s, 88.0 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:19,401::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [34b757f9] >Thread-49099::DEBUG::2014-10-23 16:49:19,401::task::595::Storage.TaskManager.Task::(_updateState) Task=`f48b03c6-b1aa-4ff3-bb45-170d9d9e1b25`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:19,401::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:49:19,401::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:19,413::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:19,413::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:19,947::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:19,947::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:19,967::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:19,967::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:19,968::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:20,133::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:20,134::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:20,134::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('2027a9ce-c630-4748-bb7c-1e8cc9c88ca2',) >Thread-49099::DEBUG::2014-10-23 16:49:20,134::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49099::INFO::2014-10-23 16:49:20,134::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:20,134::task::1191::Storage.TaskManager.Task::(prepare) Task=`f48b03c6-b1aa-4ff3-bb45-170d9d9e1b25`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:20,134::task::595::Storage.TaskManager.Task::(_updateState) Task=`f48b03c6-b1aa-4ff3-bb45-170d9d9e1b25`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:20,135::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:20,135::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:20,135::task::993::Storage.TaskManager.Task::(_decref) Task=`f48b03c6-b1aa-4ff3-bb45-170d9d9e1b25`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:20,149::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [34b757f9] >Thread-49098::DEBUG::2014-10-23 16:49:20,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:20,150::logUtils::44::dispatcher::(wrapper) Run and protect: detachStorageDomain(sdUUID='2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=2, options=None) >Thread-49098::DEBUG::2014-10-23 16:49:20,150::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`933f1ccc-e5b8-4414-93d0-3b3321800c5f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '814' at 'detachStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:20,150::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:20,150::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:20,151::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`933f1ccc-e5b8-4414-93d0-3b3321800c5f`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:20,151::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:20,151::task::993::Storage.TaskManager.Task::(_decref) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:20,151::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`ce93b3d6-ea09-4cb5-8271-2dcaf8d761fe`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '815' at 'detachStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:20,151::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:20,151::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:20,151::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`ce93b3d6-ea09-4cb5-8271-2dcaf8d761fe`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:20,152::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::_resourcesAcquired: Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:20,152::task::993::Storage.TaskManager.Task::(_decref) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:20,152::sp::954::Storage.StoragePool::(detachSD) sdUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49098::DEBUG::2014-10-23 16:49:20,152::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49098::DEBUG::2014-10-23 16:49:20,152::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:20,152::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49098::DEBUG::2014-10-23 16:49:20,152::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:20,152::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49098::DEBUG::2014-10-23 16:49:20,152::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,187::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:20,193::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,280::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,280::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,281::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:20,281::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,282::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,382::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,382::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,383::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:49:20,383::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata count=1 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,389::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00056731 s, 3.6 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,389::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00056731 s, 3.6 MB/s'], size: 2048 >Thread-49098::DEBUG::2014-10-23 16:49:20,389::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-49098::DEBUG::2014-10-23 16:49:20,390::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-49098::DEBUG::2014-10-23 16:49:20,390::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-49098::DEBUG::2014-10-23 16:49:20,390::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,390::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,390::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,390::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,390::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,391::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,485::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,486::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,486::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', '_SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe'] >Thread-49098::INFO::2014-10-23 16:49:20,486::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_imageNS already registered >Thread-49098::INFO::2014-10-23 16:49:20,487::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_volumeNS already registered >Thread-49098::INFO::2014-10-23 16:49:20,487::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_lvmActivationNS already registered >Thread-49098::INFO::2014-10-23 16:49:20,487::sd::533::Storage.StorageDomain::(detach) detaching storage domain 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 from pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49098::DEBUG::2014-10-23 16:49:20,487::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,487::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,487::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,487::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,488::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,488::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,582::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,582::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', '_SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe'] >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-49098::DEBUG::2014-10-23 16:49:20,583::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'PHYBLKSIZE=512', 'POOL_UUID=', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', 'ROLE=Regular', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', '_SHA_CKSUM=135789c42cef570033cf7f9fa0c93b868c849af6'] >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,584::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:20,680::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:20,680::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:20,681::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=135789c42cef570033cf7f9fa0c93b868c849af6, MDT_POOL_UUID= removing=MDT__SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e, MDT_POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49098::DEBUG::2014-10-23 16:49:20,681::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT__SHA_CKSUM=0db1fbe352ad81a1ec5821aa919cb7becd8ae41e --deltag MDT_POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe --addtag MDT_POOL_UUID= --addtag MDT__SHA_CKSUM=135789c42cef570033cf7f9fa0c93b868c849af6 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:20,845::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:20,885::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115273 s, 88.8 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:21,258::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:21,259::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:21,259::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:21,259::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-49098::WARNING::2014-10-23 16:49:21,259::sp::927::Storage.StoragePool::(forcedDetachSD) Force detaching domain `2027a9ce-c630-4748-bb7c-1e8cc9c88ca2` >Thread-49098::INFO::2014-10-23 16:49:21,259::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49098::DEBUG::2014-10-23 16:49:21,259::sp::1143::Storage.StoragePool::(_cleanupDomainLinks) Removing: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49098::DEBUG::2014-10-23 16:49:21,259::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49098::DEBUG::2014-10-23 16:49:21,260::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:21,260::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:21,260::sp::944::Storage.StoragePool::(forcedDetachSD) Force detach for domain `2027a9ce-c630-4748-bb7c-1e8cc9c88ca2` is done >Thread-49098::INFO::2014-10-23 16:49:21,260::logUtils::47::dispatcher::(wrapper) Run and protect: detachStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:49:21,260::task::1191::Storage.TaskManager.Task::(prepare) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::finished: None >Thread-49098::DEBUG::2014-10-23 16:49:21,260::task::595::Storage.TaskManager.Task::(_updateState) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:21,260::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': < ResourceRef 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:21,260::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:21,261::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:21,261::task::993::Storage.TaskManager.Task::(_decref) Task=`3cd0da7f-9d63-4d9f-a120-d5f70fc83f2b`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:21,275::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:49:21,275::task::595::Storage.TaskManager.Task::(_updateState) Task=`65dd938c-8abb-4997-b11c-33ca990aa972`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:21,275::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'}, options=None) >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`f2f027a1-bb9a-4a08-9201-875fe4d9aa63`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`f2f027a1-bb9a-4a08-9201-875fe4d9aa63`::Granted request >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`562cf1a1-2190-4d28-a288-9384ec9a48c6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:49:21,276::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`562cf1a1-2190-4d28-a288-9384ec9a48c6`::Granted request >Thread-49099::DEBUG::2014-10-23 16:49:21,277::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'active'} >Thread-49099::INFO::2014-10-23 16:49:21,277::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'748c2d80-0413-4285-ae64-fade24166b4b': 'Active', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,277::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,278::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:49:21,278::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:21,278::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:49:21,278::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:21,278::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:49:21,278::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:21,291::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:21,292::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:21,366::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:21,367::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,367::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,367::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,367::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,368::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,368::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,368::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) (50607) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,370::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,371::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,371::__init__::232::IOProcess::(_processLogs) (50608) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:21,371::__init__::232::IOProcess::(_processLogs) (50607) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:21,371::__init__::232::IOProcess::(_processLogs) (50608) Queuing response >Thread-49099::DEBUG::2014-10-23 16:49:21,374::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:49:21,374::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,375::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,375::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,375::__init__::232::IOProcess::(_processLogs) (50609) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:21,376::__init__::232::IOProcess::(_processLogs) (50609) Queuing response >Thread-49099::DEBUG::2014-10-23 16:49:21,376::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:21,376::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,376::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,376::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,377::__init__::232::IOProcess::(_processLogs) (50610) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:21,377::__init__::232::IOProcess::(_processLogs) (50610) Queuing response >Thread-49099::DEBUG::2014-10-23 16:49:21,377::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) (50611) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) (50611) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,378::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,379::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,379::__init__::232::IOProcess::(_processLogs) (50612) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:21,379::__init__::232::IOProcess::(_processLogs) (50612) Queuing response >Thread-49099::DEBUG::2014-10-23 16:49:21,379::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49099::INFO::2014-10-23 16:49:21,379::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49099::INFO::2014-10-23 16:49:21,379::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49099::DEBUG::2014-10-23 16:49:21,379::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-49099::DEBUG::2014-10-23 16:49:21,379::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49099::DEBUG::2014-10-23 16:49:21,380::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:21,380::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:21,380::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49099::WARNING::2014-10-23 16:49:21,380::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe already exists >Thread-49099::DEBUG::2014-10-23 16:49:21,380::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:21,381::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 748c2d80-0413-4285-ae64-fade24166b4b (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:21,475::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found\n Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b\n'; <rc> = 5 >Thread-49099::WARNING::2014-10-23 16:49:21,476::lvm::370::Storage.LVM::(_reloadvgs) lvm vgs failed: 5 [] [' Volume group "748c2d80-0413-4285-ae64-fade24166b4b" not found', ' Skipping volume group 748c2d80-0413-4285-ae64-fade24166b4b'] >Thread-49099::DEBUG::2014-10-23 16:49:21,476::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:49:21,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,478::__init__::232::IOProcess::(_processLogs) (50613) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:21,479::__init__::232::IOProcess::(_processLogs) (50613) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:21,479::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:21,479::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:21,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:21,480::__init__::232::IOProcess::(_processLogs) (50614) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:21,480::__init__::232::IOProcess::(_processLogs) (50614) Queuing response >Thread-49099::INFO::2014-10-23 16:49:21,480::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::DEBUG::2014-10-23 16:49:21,480::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49099::INFO::2014-10-23 16:49:21,480::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::DEBUG::2014-10-23 16:49:21,481::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49099::INFO::2014-10-23 16:49:21,481::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:49:21,481::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49099::DEBUG::2014-10-23 16:49:21,481::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49099::DEBUG::2014-10-23 16:49:21,481::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:49:21,481::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:49:21,481::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49099::DEBUG::2014-10-23 16:49:21,481::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49099::DEBUG::2014-10-23 16:49:21,482::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:49:21,482::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:49:21,482::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49099::INFO::2014-10-23 16:49:21,482::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49099::DEBUG::2014-10-23 16:49:21,482::task::1191::Storage.TaskManager.Task::(prepare) Task=`65dd938c-8abb-4997-b11c-33ca990aa972`::finished: True >Thread-49099::DEBUG::2014-10-23 16:49:21,482::task::595::Storage.TaskManager.Task::(_updateState) Task=`65dd938c-8abb-4997-b11c-33ca990aa972`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:21,482::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:21,482::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:21,482::task::993::Storage.TaskManager.Task::(_decref) Task=`65dd938c-8abb-4997-b11c-33ca990aa972`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:21,553::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:49:21,553::task::595::Storage.TaskManager.Task::(_updateState) Task=`32696e25-7b63-4590-b287-5da32b565e9e`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:21,554::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:49:21,558::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -r 2 -u (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:22,121::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:22,121::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:49:22,122::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:22,122::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:49:22,122::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:22,122::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:49:22,122::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:22,132::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:22,133::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:22,227::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:22,227::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:22,228::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::INFO::2014-10-23 16:49:22,228::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:22,229::task::1191::Storage.TaskManager.Task::(prepare) Task=`32696e25-7b63-4590-b287-5da32b565e9e`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:22,229::task::595::Storage.TaskManager.Task::(_updateState) Task=`32696e25-7b63-4590-b287-5da32b565e9e`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:22,229::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:22,229::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:22,229::task::993::Storage.TaskManager.Task::(_decref) Task=`32696e25-7b63-4590-b287-5da32b565e9e`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:22,894::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:22,935::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117606 s, 87.1 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:24,944::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:24,985::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011689 s, 87.6 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:26,470::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [12076440] >Thread-49099::DEBUG::2014-10-23 16:49:26,471::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bd83b3e-8cea-4205-a12c-f2847e22dcfb`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:26,471::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:49:26,471::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:26,482::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:26,482::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-41::DEBUG::2014-10-23 16:49:26,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,660::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,660::__init__::232::IOProcess::(_processLogs) (50615) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:26,661::__init__::232::IOProcess::(_processLogs) (50615) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:26,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,661::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,662::__init__::232::IOProcess::(_processLogs) (50616) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:26,662::__init__::232::IOProcess::(_processLogs) (50616) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:26,663::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:49:26,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,663::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,664::__init__::232::IOProcess::(_processLogs) (50617) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:26,664::__init__::232::IOProcess::(_processLogs) (50617) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:26,665::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:26,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,665::__init__::232::IOProcess::(_processLogs) (50618) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:26,665::__init__::232::IOProcess::(_processLogs) (50618) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:26,666::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-41::DEBUG::2014-10-23 16:49:26,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,666::__init__::232::IOProcess::(_processLogs) (50619) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) (50619) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) (50620) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:26,667::__init__::232::IOProcess::(_processLogs) (50620) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:26,668::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49139::INFO::2014-10-23 16:49:26,668::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49139::INFO::2014-10-23 16:49:26,668::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:49:26,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,668::__init__::232::IOProcess::(_processLogs) (50621) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:26,669::__init__::232::IOProcess::(_processLogs) (50621) Queuing response >Thread-49139::DEBUG::2014-10-23 16:49:26,669::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49139::DEBUG::2014-10-23 16:49:26,675::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n453 bytes (453 B) copied, 0.000372515 s, 1.2 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:26,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:26,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:26,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:26,676::__init__::232::IOProcess::(_processLogs) (50622) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:26,676::__init__::232::IOProcess::(_processLogs) (50622) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:26,687::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49192::DEBUG::2014-10-23 16:49:26,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`1117f4dd-3c49-4e83-b3c9-7db49e526711`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:26,688::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::INFO::2014-10-23 16:49:26,699::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:26,699::task::1191::Storage.TaskManager.Task::(prepare) Task=`1117f4dd-3c49-4e83-b3c9-7db49e526711`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:26,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`1117f4dd-3c49-4e83-b3c9-7db49e526711`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:26,699::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:26,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:26,700::task::993::Storage.TaskManager.Task::(_decref) Task=`1117f4dd-3c49-4e83-b3c9-7db49e526711`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:26,710::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:26,710::task::595::Storage.TaskManager.Task::(_updateState) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:26,710::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:49:26,710::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a6ed137e-b249-4d69-b8ba-17563d10f64f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:26,710::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:26,711::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:26,711::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a6ed137e-b249-4d69-b8ba-17563d10f64f`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:26,711::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:26,711::task::993::Storage.TaskManager.Task::(_decref) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:26,724::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:26,724::task::1191::Storage.TaskManager.Task::(prepare) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Active,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:26,724::task::595::Storage.TaskManager.Task::(_updateState) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:26,724::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:26,725::task::993::Storage.TaskManager.Task::(_decref) Task=`22ef17cd-60a0-498c-b8c0-69de84f8b9ac`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:26,994::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,007::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,008::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:27,045::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188026 s, 54.5 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,046::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,047::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,051::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-41::DEBUG::2014-10-23 16:49:27,111::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:27,112::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:27,112::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:27,112::__init__::232::IOProcess::(_processLogs) (50623) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:27,113::__init__::232::IOProcess::(_processLogs) (50623) Queuing response >Thread-49080::DEBUG::2014-10-23 16:49:27,114::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:49:27,124::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000295652 s, 1.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:27,125::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:27,125::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:27,125::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:27,125::__init__::232::IOProcess::(_processLogs) (50624) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:27,125::__init__::232::IOProcess::(_processLogs) (50624) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:27,126::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:27,126::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:27,126::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:27,126::__init__::232::IOProcess::(_processLogs) (50625) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) (50625) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) (50626) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:27,127::__init__::232::IOProcess::(_processLogs) (50626) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:27,128::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:27,128::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:27,128::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:27,128::__init__::232::IOProcess::(_processLogs) (50627) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:27,128::__init__::232::IOProcess::(_processLogs) (50627) Queuing response >Thread-49099::DEBUG::2014-10-23 16:49:27,196::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,196::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,196::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', '2d4b6e8f-878f-4d9e-96f5-176b679b43e4') >Thread-49099::DEBUG::2014-10-23 16:49:27,196::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49099::INFO::2014-10-23 16:49:27,196::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:27,197::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bd83b3e-8cea-4205-a12c-f2847e22dcfb`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:27,197::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bd83b3e-8cea-4205-a12c-f2847e22dcfb`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:27,197::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:27,197::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:27,197::task::993::Storage.TaskManager.Task::(_decref) Task=`4bd83b3e-8cea-4205-a12c-f2847e22dcfb`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:27,203::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [12076440] >Thread-49099::DEBUG::2014-10-23 16:49:27,204::task::595::Storage.TaskManager.Task::(_updateState) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:27,204::logUtils::44::dispatcher::(wrapper) Run and protect: formatStorageDomain(sdUUID='2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', autoDetach=False, options=None) >Thread-49099::DEBUG::2014-10-23 16:49:27,204::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:49:27,204::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:27,204::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:49:27,204::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,224::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:27,225::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,306::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,307::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`5b6b0ea2-ec3f-410f-80a4-e2dc65facee4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2747' at 'formatStorageDomain' >Thread-49099::DEBUG::2014-10-23 16:49:27,307::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' for lock type 'exclusive' >Thread-49099::DEBUG::2014-10-23 16:49:27,307::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free. Now locking as 'exclusive' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:49:27,307::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2`ReqID=`5b6b0ea2-ec3f-410f-80a4-e2dc65facee4`::Granted request >Thread-49099::DEBUG::2014-10-23 16:49:27,307::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::_resourcesAcquired: Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (exclusive) >Thread-49099::DEBUG::2014-10-23 16:49:27,308::task::993::Storage.TaskManager.Task::(_decref) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::ref 1 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:27,308::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:49:27,308::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:27,308::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:49:27,308::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:27,308::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:49:27,308::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,321::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:27,322::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,392::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,392::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,392::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,392::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,393::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,393::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,393::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,393::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:27,393::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,394::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:27,412::task::595::Storage.TaskManager.Task::(_updateState) Task=`9803235c-333b-469d-8fb2-d9d3b7070de0`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:27,412::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:49:27,412::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000372515', 'lastCheck': '0.7', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295652', 'lastCheck': '0.3', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:27,413::task::1191::Storage.TaskManager.Task::(prepare) Task=`9803235c-333b-469d-8fb2-d9d3b7070de0`::finished: {'748c2d80-0413-4285-ae64-fade24166b4b': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000372515', 'lastCheck': '0.7', 'valid': True}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000295652', 'lastCheck': '0.3', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:27,413::task::595::Storage.TaskManager.Task::(_updateState) Task=`9803235c-333b-469d-8fb2-d9d3b7070de0`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:27,413::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:27,413::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:27,413::task::993::Storage.TaskManager.Task::(_decref) Task=`9803235c-333b-469d-8fb2-d9d3b7070de0`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:27,501::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,501::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,501::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-49099::DEBUG::2014-10-23 16:49:27,501::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata count=1 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,508::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000865677 s, 2.4 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,508::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000865677 s, 2.4 MB/s'], size: 2048 >Thread-49099::DEBUG::2014-10-23 16:49:27,508::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-49099::DEBUG::2014-10-23 16:49:27,509::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-49099::DEBUG::2014-10-23 16:49:27,509::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-49099::DEBUG::2014-10-23 16:49:27,509::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,509::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,509::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,509::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,510::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,510::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,607::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,607::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,608::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', 'POOL_UUID=', '_SHA_CKSUM=135789c42cef570033cf7f9fa0c93b868c849af6'] >Thread-49099::INFO::2014-10-23 16:49:27,608::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_imageNS already registered >Thread-49099::INFO::2014-10-23 16:49:27,608::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_volumeNS already registered >Thread-49099::INFO::2014-10-23 16:49:27,608::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2_lvmActivationNS already registered >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,609::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,709::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,709::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,709::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'ROLE=Regular', 'CLASS=Data', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'VGUUID=bdt138-XHXP-bJao-PkcY-Aj58-ed4i-kLe3yL', 'PHYBLKSIZE=512', 'DESCRIPTION=iscsi_0', 'SDUUID=2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', u'PV0=pv:1cmg02,uuid:50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ,pestart:0,pecount:757,mapoffset:0', 'POOL_UUID=', '_SHA_CKSUM=135789c42cef570033cf7f9fa0c93b868c849af6'] >Thread-49099::DEBUG::2014-10-23 16:49:27,709::fileUtils::124::Storage.fileUtils::(cleanupdir) Removing directory: 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 >Thread-49099::DEBUG::2014-10-23 16:49:27,710::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,710::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,806::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,806::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-49099::DEBUG::2014-10-23 16:49:27,806::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:27,806::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/leases >Thread-49099::DEBUG::2014-10-23 16:49:27,807::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/leases (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:27,934::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:27,934::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:27,934::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:27,935::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/inbox >Thread-49099::DEBUG::2014-10-23 16:49:27,935::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/inbox (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,053::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,053::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,053::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:28,053::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/master >Thread-49099::DEBUG::2014-10-23 16:49:28,054::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/master (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,177::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,177::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,177::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:28,178::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/outbox >Thread-49099::DEBUG::2014-10-23 16:49:28,178::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/outbox (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,305::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,305::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,305::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:28,306::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/ids >Thread-49099::DEBUG::2014-10-23 16:49:28,306::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/ids (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,419::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,419::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,420::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::WARNING::2014-10-23 16:49:28,420::lvm::1087::Storage.LVM::(removeLVs) Removing active volume 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata >Thread-49099::DEBUG::2014-10-23 16:49:28,420::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f --autobackup n 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2/metadata (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,735::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,735::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,736::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,736::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgremove --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -f 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2 (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:28,842::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:28,842::lvm::483::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:28,842::lvm::486::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::INFO::2014-10-23 16:49:28,843::logUtils::47::dispatcher::(wrapper) Run and protect: formatStorageDomain, Return response: None >Thread-49099::DEBUG::2014-10-23 16:49:28,846::task::1191::Storage.TaskManager.Task::(prepare) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::finished: None >Thread-49099::DEBUG::2014-10-23 16:49:28,846::task::595::Storage.TaskManager.Task::(_updateState) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:28,846::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2': < ResourceRef 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', isValid: 'True' obj: 'None'>} >Thread-49099::DEBUG::2014-10-23 16:49:28,847::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:28,847::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' >Thread-49099::DEBUG::2014-10-23 16:49:28,847::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' (0 active users) >Thread-49099::DEBUG::2014-10-23 16:49:28,847::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2' is free, finding out if anyone is waiting for it. >Thread-49099::DEBUG::2014-10-23 16:49:28,847::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.2027a9ce-c630-4748-bb7c-1e8cc9c88ca2', Clearing records. >Thread-49099::DEBUG::2014-10-23 16:49:28,847::task::993::Storage.TaskManager.Task::(_decref) Task=`a972d486-f7af-4ebd-a688-6039055dd26b`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:28,914::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [12076440] >Thread-49099::DEBUG::2014-10-23 16:49:28,914::task::595::Storage.TaskManager.Task::(_updateState) Task=`71e58f0c-9e6d-4f28-a51d-c166f05e4c25`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:28,914::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b', 'port': '3260'}], options=None) >Thread-49099::DEBUG::2014-10-23 16:49:28,915::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -r 3 -u (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:29,070::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:29,117::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0177395 s, 57.7 MB/s\n'; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:29,482::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:29,482::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49099::DEBUG::2014-10-23 16:49:29,482::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:29,482::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49099::DEBUG::2014-10-23 16:49:29,482::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49099::DEBUG::2014-10-23 16:49:29,483::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49099::DEBUG::2014-10-23 16:49:29,483::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:29,496::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::DEBUG::2014-10-23 16:49:29,497::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49099::DEBUG::2014-10-23 16:49:29,561::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49099::DEBUG::2014-10-23 16:49:29,561::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49099::DEBUG::2014-10-23 16:49:29,562::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49099::INFO::2014-10-23 16:49:29,562::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:29,563::task::1191::Storage.TaskManager.Task::(prepare) Task=`71e58f0c-9e6d-4f28-a51d-c166f05e4c25`::finished: {'statuslist': [{'status': 0, 'id': '3820076e-e4d2-4093-8353-43eb6dbde15b'}]} >Thread-49099::DEBUG::2014-10-23 16:49:29,563::task::595::Storage.TaskManager.Task::(_updateState) Task=`71e58f0c-9e6d-4f28-a51d-c166f05e4c25`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:29,563::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:29,563::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:29,563::task::993::Storage.TaskManager.Task::(_decref) Task=`71e58f0c-9e6d-4f28-a51d-c166f05e4c25`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:31,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:31,168::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0132684 s, 77.2 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:33,177::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:33,217::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116722 s, 87.7 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:35,016::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [47b1b58a] >Thread-49098::DEBUG::2014-10-23 16:49:35,016::task::595::Storage.TaskManager.Task::(_updateState) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:35,016::logUtils::44::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID='748c2d80-0413-4285-ae64-fade24166b4b', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=2, options=None) >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d42207a4-f2d2-46a5-822e-fc3d750479be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1227' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`d42207a4-f2d2-46a5-822e-fc3d750479be`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:35,017::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:35,017::task::993::Storage.TaskManager.Task::(_decref) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`4dec2d4e-fc4f-4f0a-a62e-60c8db54c1dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1228' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:35,017::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:35,018::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:35,018::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`4dec2d4e-fc4f-4f0a-a62e-60c8db54c1dd`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:35,018::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::_resourcesAcquired: Storage.748c2d80-0413-4285-ae64-fade24166b4b (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:35,018::task::993::Storage.TaskManager.Task::(_decref) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:35,018::sp::1070::Storage.StoragePool::(deactivateSD) sdUUID=748c2d80-0413-4285-ae64-fade24166b4b spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-49098::INFO::2014-10-23 16:49:35,020::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'748c2d80-0413-4285-ae64-fade24166b4b': 'Attached', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49098::DEBUG::2014-10-23 16:49:35,020::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49098::DEBUG::2014-10-23 16:49:35,020::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::INFO::2014-10-23 16:49:35,020::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 748c2d80-0413-4285-ae64-fade24166b4b >Thread-49139::DEBUG::2014-10-23 16:49:35,020::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 748c2d80-0413-4285-ae64-fade24166b4b >Thread-49139::INFO::2014-10-23 16:49:35,021::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 748c2d80-0413-4285-ae64-fade24166b4b (id: 1) >Dummy-2584::DEBUG::2014-10-23 16:49:35,226::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:35,268::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116881 s, 87.6 MB/s\n'; <rc> = 0 >Thread-49139::DEBUG::2014-10-23 16:49:37,022::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 748c2d80-0413-4285-ae64-fade24166b4b released successfully (id: 1) >Thread-49098::DEBUG::2014-10-23 16:49:37,022::sp::1407::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `11cfd6be-5798-4973-9e8b-a3262a2d3ebe` stopped monitoring domain `748c2d80-0413-4285-ae64-fade24166b4b` >Thread-49098::DEBUG::2014-10-23 16:49:37,022::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::INFO::2014-10-23 16:49:37,022::logUtils::47::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:49:37,022::task::1191::Storage.TaskManager.Task::(prepare) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::finished: None >Thread-49098::DEBUG::2014-10-23 16:49:37,023::task::595::Storage.TaskManager.Task::(_updateState) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.748c2d80-0413-4285-ae64-fade24166b4b': < ResourceRef 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:37,023::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:37,024::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:37,024::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:37,024::task::993::Storage.TaskManager.Task::(_decref) Task=`133b762f-4bd2-4cb8-b67c-296256d305ee`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:49:37,039::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49099::DEBUG::2014-10-23 16:49:37,040::task::595::Storage.TaskManager.Task::(_updateState) Task=`75bb3549-1c34-4ccd-a8d4-5b686096ab91`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:49:37,040::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49099::INFO::2014-10-23 16:49:37,052::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:37,052::task::1191::Storage.TaskManager.Task::(prepare) Task=`75bb3549-1c34-4ccd-a8d4-5b686096ab91`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49099::DEBUG::2014-10-23 16:49:37,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`75bb3549-1c34-4ccd-a8d4-5b686096ab91`::moving from state preparing -> state finished >Thread-49099::DEBUG::2014-10-23 16:49:37,052::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49099::DEBUG::2014-10-23 16:49:37,052::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49099::DEBUG::2014-10-23 16:49:37,052::task::993::Storage.TaskManager.Task::(_decref) Task=`75bb3549-1c34-4ccd-a8d4-5b686096ab91`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:37,054::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49192::DEBUG::2014-10-23 16:49:37,054::task::595::Storage.TaskManager.Task::(_updateState) Task=`457bf3d7-740b-4bb2-bf2e-ccce9093064c`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:37,054::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'attached'}, options=None) >Thread-49192::DEBUG::2014-10-23 16:49:37,054::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`e32e48b8-bdf5-4fa5-9939-01ee206074e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:49:37,054::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:49:37,054::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:49:37,055::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`e32e48b8-bdf5-4fa5-9939-01ee206074e0`::Granted request >Thread-49192::DEBUG::2014-10-23 16:49:37,055::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7adf8bc6-c2b0-45f7-b919-85cf0577c236`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:49:37,055::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49192::DEBUG::2014-10-23 16:49:37,055::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:49:37,055::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7adf8bc6-c2b0-45f7-b919-85cf0577c236`::Granted request >Thread-49192::DEBUG::2014-10-23 16:49:37,055::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active', '748c2d80-0413-4285-ae64-fade24166b4b': 'attached'} >Thread-49192::INFO::2014-10-23 16:49:37,055::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'748c2d80-0413-4285-ae64-fade24166b4b': 'Attached', '8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,056::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:37,056::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:37,056::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:37,057::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:37,057::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:37,057::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:37,068::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:37,069::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:37,070::task::595::Storage.TaskManager.Task::(_updateState) Task=`605153e1-5170-4365-9714-984faf906641`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:37,070::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::DEBUG::2014-10-23 16:49:37,070::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:37,077::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`615f98d2-7f81-4e8d-9845-a2120e804f2f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:37,077::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:37,077::resourceManager::566::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' found in shared state and queue is empty, Joining current shared lock (2 active users) >Thread-49098::DEBUG::2014-10-23 16:49:37,077::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`615f98d2-7f81-4e8d-9845-a2120e804f2f`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:37,077::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`605153e1-5170-4365-9714-984faf906641`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:37,077::task::993::Storage.TaskManager.Task::(_decref) Task=`605153e1-5170-4365-9714-984faf906641`::ref 1 aborting False >Thread-49080::DEBUG::2014-10-23 16:49:37,129::domainMonitor::209::Storage.DomainMonitorThread::(_monitorDomain) Refreshing domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:49:37,144::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,145::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:37,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,147::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,147::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,147::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,148::__init__::232::IOProcess::(_processLogs) (50628) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:37,148::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,148::__init__::232::IOProcess::(_processLogs) (50629) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:37,148::__init__::232::IOProcess::(_processLogs) (50629) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:37,148::__init__::232::IOProcess::(_processLogs) (50628) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,150::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:49:37,150::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,150::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,150::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,150::__init__::232::IOProcess::(_processLogs) (50630) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:37,151::__init__::232::IOProcess::(_processLogs) (50630) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,151::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:37,152::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,152::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,152::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,152::__init__::232::IOProcess::(_processLogs) (50631) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:37,152::__init__::232::IOProcess::(_processLogs) (50631) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,153::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:49:37,153::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,153::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,153::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,154::__init__::232::IOProcess::(_processLogs) (50632) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:37,157::__init__::232::IOProcess::(_processLogs) (50632) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:37,158::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,159::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,159::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,159::__init__::232::IOProcess::(_processLogs) (50633) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:37,159::__init__::232::IOProcess::(_processLogs) (50633) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,159::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49192::INFO::2014-10-23 16:49:37,159::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49192::INFO::2014-10-23 16:49:37,160::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49192::DEBUG::2014-10-23 16:49:37,161::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-41::DEBUG::2014-10-23 16:49:37,163::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-49192::DEBUG::2014-10-23 16:49:37,163::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-41::DEBUG::2014-10-23 16:49:37,165::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-49192::DEBUG::2014-10-23 16:49:37,165::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-41::DEBUG::2014-10-23 16:49:37,167::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-49192::DEBUG::2014-10-23 16:49:37,167::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:37,169::__init__::232::IOProcess::(_processLogs) (50634) Got request for method 'statvfs' >Thread-49192::DEBUG::2014-10-23 16:49:37,169::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49192::WARNING::2014-10-23 16:49:37,170::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe already exists >Thread-41::DEBUG::2014-10-23 16:49:37,170::__init__::232::IOProcess::(_processLogs) (50634) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,172::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49080::DEBUG::2014-10-23 16:49:37,176::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:37,182::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-49098::INFO::2014-10-23 16:49:37,189::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Attached,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49080::DEBUG::2014-10-23 16:49:37,189::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000335585 s, 957 kB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:37,190::task::1191::Storage.TaskManager.Task::(prepare) Task=`605153e1-5170-4365-9714-984faf906641`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Attached,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:37,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`605153e1-5170-4365-9714-984faf906641`::moving from state preparing -> state finished >Thread-41::DEBUG::2014-10-23 16:49:37,191::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-49098::DEBUG::2014-10-23 16:49:37,191::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-41::DEBUG::2014-10-23 16:49:37,192::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-49098::DEBUG::2014-10-23 16:49:37,192::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-41::DEBUG::2014-10-23 16:49:37,193::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,194::__init__::232::IOProcess::(_processLogs) (50635) Got request for method 'statvfs' >Thread-49098::DEBUG::2014-10-23 16:49:37,194::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-41::DEBUG::2014-10-23 16:49:37,194::__init__::232::IOProcess::(_processLogs) (50635) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:37,195::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (1 active users) >Thread-49098::DEBUG::2014-10-23 16:49:37,197::task::993::Storage.TaskManager.Task::(_decref) Task=`605153e1-5170-4365-9714-984faf906641`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:49:37,199::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,200::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,201::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,201::__init__::232::IOProcess::(_processLogs) (50636) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:37,202::__init__::232::IOProcess::(_processLogs) (50636) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:37,202::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) (50637) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) (50637) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,203::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,204::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,204::__init__::232::IOProcess::(_processLogs) (50638) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:37,204::__init__::232::IOProcess::(_processLogs) (50638) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:37,276::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:37,276::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) (50639) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) (50639) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:37,278::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:37,279::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:37,279::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:37,279::__init__::232::IOProcess::(_processLogs) (50640) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:37,279::__init__::232::IOProcess::(_processLogs) (50640) Queuing response >Thread-49192::INFO::2014-10-23 16:49:37,280::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:49:37,280::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::INFO::2014-10-23 16:49:37,280::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:49:37,280::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:49:37,280::sp::1221::Storage.StoragePool::(__rebuild) collecting stale storage domain link /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:49:37,281::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49192::INFO::2014-10-23 16:49:37,281::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49192::DEBUG::2014-10-23 16:49:37,282::task::1191::Storage.TaskManager.Task::(prepare) Task=`457bf3d7-740b-4bb2-bf2e-ccce9093064c`::finished: True >Thread-49192::DEBUG::2014-10-23 16:49:37,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`457bf3d7-740b-4bb2-bf2e-ccce9093064c`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:37,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:37,282::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:37,282::task::993::Storage.TaskManager.Task::(_decref) Task=`457bf3d7-740b-4bb2-bf2e-ccce9093064c`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:37,320::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [7d2b03c7] >Thread-49192::DEBUG::2014-10-23 16:49:37,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c6c2be-7b3a-4ea6-a23c-007b0aa748df`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:37,320::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a811e94-b897-42aa-bdf1-3067374bd758', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:49:37,321::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg01 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:37,365::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:37,366::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:37,366::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:37,366::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:37,366::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:37,366::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:37,377::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:37,378::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:37,413::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:37,458::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117732 s, 87.0 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:37,459::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:37,460::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,461::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,461::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,462::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,462::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,463::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:37,464::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:49:37,464::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:37,464::task::1191::Storage.TaskManager.Task::(prepare) Task=`16c6c2be-7b3a-4ea6-a23c-007b0aa748df`::finished: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:37,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`16c6c2be-7b3a-4ea6-a23c-007b0aa748df`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:37,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:37,466::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:37,466::task::993::Storage.TaskManager.Task::(_decref) Task=`16c6c2be-7b3a-4ea6-a23c-007b0aa748df`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:39,476::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:39,517::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122459 s, 83.6 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:41,526::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:41,567::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0119874 s, 85.4 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:43,190::task::595::Storage.TaskManager.Task::(_updateState) Task=`49105b25-083a-45f8-9de0-7aba8b975eb5`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:43,190::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:49:43,191::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000335585', 'lastCheck': '6.0', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:43,191::task::1191::Storage.TaskManager.Task::(prepare) Task=`49105b25-083a-45f8-9de0-7aba8b975eb5`::finished: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000335585', 'lastCheck': '6.0', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:43,191::task::595::Storage.TaskManager.Task::(_updateState) Task=`49105b25-083a-45f8-9de0-7aba8b975eb5`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:43,191::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:43,191::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:43,191::task::993::Storage.TaskManager.Task::(_decref) Task=`49105b25-083a-45f8-9de0-7aba8b975eb5`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:43,576::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:43,623::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0182242 s, 56.2 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:45,632::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:45,677::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0158918 s, 64.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:47,206::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:47,209::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:47,209::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:47,209::__init__::232::IOProcess::(_processLogs) (50641) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:47,209::__init__::232::IOProcess::(_processLogs) (50641) Queuing response >Thread-49080::DEBUG::2014-10-23 16:49:47,210::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:49:47,216::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000263712 s, 1.2 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:47,218::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:47,218::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:47,218::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:47,218::__init__::232::IOProcess::(_processLogs) (50642) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:47,218::__init__::232::IOProcess::(_processLogs) (50642) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:47,219::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:47,219::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:47,219::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:47,219::__init__::232::IOProcess::(_processLogs) (50643) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:47,219::__init__::232::IOProcess::(_processLogs) (50643) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:47,220::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:47,220::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:47,220::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:47,220::__init__::232::IOProcess::(_processLogs) (50644) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:47,220::__init__::232::IOProcess::(_processLogs) (50644) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:47,221::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:47,221::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:47,221::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:47,221::__init__::232::IOProcess::(_processLogs) (50645) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:47,221::__init__::232::IOProcess::(_processLogs) (50645) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:47,284::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49192::DEBUG::2014-10-23 16:49:47,284::task::595::Storage.TaskManager.Task::(_updateState) Task=`de314e8e-98aa-45f7-ad1f-9190fe2b3db0`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:47,285::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::INFO::2014-10-23 16:49:47,296::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:47,296::task::1191::Storage.TaskManager.Task::(prepare) Task=`de314e8e-98aa-45f7-ad1f-9190fe2b3db0`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:47,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`de314e8e-98aa-45f7-ad1f-9190fe2b3db0`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:47,296::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:47,296::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:47,296::task::993::Storage.TaskManager.Task::(_decref) Task=`de314e8e-98aa-45f7-ad1f-9190fe2b3db0`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:47,306::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:47,306::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:47,307::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:49:47,307::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9fd96855-1d03-4da0-85ef-8d99fc4b2217`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:47,307::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:47,307::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:47,307::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9fd96855-1d03-4da0-85ef-8d99fc4b2217`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:47,307::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:47,308::task::993::Storage.TaskManager.Task::(_decref) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:47,332::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Attached,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:47,332::task::1191::Storage.TaskManager.Task::(prepare) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '748c2d80-0413-4285-ae64-fade24166b4b:Attached,8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'748c2d80-0413-4285-ae64-fade24166b4b': {'status': 'Attached', 'isoprefix': '', 'alerts': []}, '8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:47,332::task::595::Storage.TaskManager.Task::(_updateState) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:47,332::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:47,332::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:47,332::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:47,332::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:47,333::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:47,333::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:47,333::task::993::Storage.TaskManager.Task::(_decref) Task=`c7528c36-b5e0-4b28-881c-069c676f7ee7`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:47,695::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:47,746::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0188837 s, 54.2 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:49,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:49,850::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0115971 s, 88.3 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:49:51,859::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:51,900::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121413 s, 84.3 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:53,551::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [2482c607] >Thread-49192::DEBUG::2014-10-23 16:49:53,551::task::595::Storage.TaskManager.Task::(_updateState) Task=`708b6ce6-1fca-4762-98ea-ed7f1f7cf680`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:53,552::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a811e94-b897-42aa-bdf1-3067374bd758', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:49:53,553::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/10.34.63.199:_cmg01 >Thread-49192::DEBUG::2014-10-23 16:49:53,553::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.34.63.199:/cmg01 /rhev/data-center/mnt/10.34.63.199:_cmg01 (cwd None) >Thread-41::DEBUG::2014-10-23 16:49:53,577::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,578::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,578::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,579::__init__::232::IOProcess::(_processLogs) (50646) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:53,579::__init__::232::IOProcess::(_processLogs) (50646) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,586::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.199:_cmg01 >Thread-41::DEBUG::2014-10-23 16:49:53,587::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,587::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,587::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,587::__init__::232::IOProcess::(_processLogs) (50647) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,587::__init__::232::IOProcess::(_processLogs) (50647) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,588::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'748c2d80-0413-4285-ae64-fade24166b4b',) >Thread-49192::DEBUG::2014-10-23 16:49:53,588::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49192::INFO::2014-10-23 16:49:53,588::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:53,588::task::1191::Storage.TaskManager.Task::(prepare) Task=`708b6ce6-1fca-4762-98ea-ed7f1f7cf680`::finished: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:53,588::task::595::Storage.TaskManager.Task::(_updateState) Task=`708b6ce6-1fca-4762-98ea-ed7f1f7cf680`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:53,588::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:53,588::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:53,588::task::993::Storage.TaskManager.Task::(_decref) Task=`708b6ce6-1fca-4762-98ea-ed7f1f7cf680`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:53,608::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [cb43d12] >Thread-49098::DEBUG::2014-10-23 16:49:53,608::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:53,608::logUtils::44::dispatcher::(wrapper) Run and protect: detachStorageDomain(sdUUID='748c2d80-0413-4285-ae64-fade24166b4b', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=2, options=None) >Thread-49098::DEBUG::2014-10-23 16:49:53,609::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7c57ee6b-c263-4134-8d4b-8f3d128b8eae`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '814' at 'detachStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:53,609::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:53,609::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:53,609::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7c57ee6b-c263-4134-8d4b-8f3d128b8eae`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:53,609::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:53,609::task::993::Storage.TaskManager.Task::(_decref) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:53,610::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`ffd9e31a-5f86-4488-9c3c-29eff2151b2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '815' at 'detachStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:49:53,610::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:49:53,610::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:53,610::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`ffd9e31a-5f86-4488-9c3c-29eff2151b2b`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:53,610::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::_resourcesAcquired: Storage.748c2d80-0413-4285-ae64-fade24166b4b (exclusive) >Thread-49098::DEBUG::2014-10-23 16:49:53,610::task::993::Storage.TaskManager.Task::(_decref) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:53,610::sp::954::Storage.StoragePool::(detachSD) sdUUID=748c2d80-0413-4285-ae64-fade24166b4b spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49098::DEBUG::2014-10-23 16:49:53,610::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49098::DEBUG::2014-10-23 16:49:53,611::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:53,611::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49098::DEBUG::2014-10-23 16:49:53,611::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:53,611::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49098::DEBUG::2014-10-23 16:49:53,611::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:53,621::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:53,622::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49098::DEBUG::2014-10-23 16:49:53,748::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49098::DEBUG::2014-10-23 16:49:53,749::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:53,752::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,752::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,752::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,752::__init__::232::IOProcess::(_processLogs) (50648) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,753::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,753::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,753::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,753::__init__::232::IOProcess::(_processLogs) (50649) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,753::__init__::232::IOProcess::(_processLogs) (50648) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:53,754::__init__::232::IOProcess::(_processLogs) (50649) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,755::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:49:53,756::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,756::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,756::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,756::__init__::232::IOProcess::(_processLogs) (50650) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:53,757::__init__::232::IOProcess::(_processLogs) (50650) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,757::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:53,758::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,758::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,758::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,758::__init__::232::IOProcess::(_processLogs) (50651) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:53,759::__init__::232::IOProcess::(_processLogs) (50651) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,759::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-41::DEBUG::2014-10-23 16:49:53,760::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,760::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,760::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,760::__init__::232::IOProcess::(_processLogs) (50652) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:53,760::__init__::232::IOProcess::(_processLogs) (50652) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:53,761::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,761::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,761::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,761::__init__::232::IOProcess::(_processLogs) (50653) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,762::__init__::232::IOProcess::(_processLogs) (50653) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,762::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49098::INFO::2014-10-23 16:49:53,762::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49098::INFO::2014-10-23 16:49:53,762::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-49098::INFO::2014-10-23 16:49:53,763::sd::533::Storage.StorageDomain::(detach) detaching storage domain 748c2d80-0413-4285-ae64-fade24166b4b from pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-41::DEBUG::2014-10-23 16:49:53,763::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,763::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,763::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,763::__init__::232::IOProcess::(_processLogs) (50654) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:53,763::__init__::232::IOProcess::(_processLogs) (50654) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=4b128bbbf740e3433fdf461620a3ae70e6ba5e5d'] >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-49098::DEBUG::2014-10-23 16:49:53,764::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f1dd9f262ee1d606554abe25e457a9443edbd96f'] >Thread-41::DEBUG::2014-10-23 16:49:53,765::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,765::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,765::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,765::__init__::232::IOProcess::(_processLogs) (50655) Got request for method 'writefile' >Thread-41::DEBUG::2014-10-23 16:49:53,767::__init__::232::IOProcess::(_processLogs) (50655) Queuing response >Thread-49098::WARNING::2014-10-23 16:49:53,767::outOfProcess::221::Storage.oop::(rename) renaming a directory is not an atomic operation >Thread-41::DEBUG::2014-10-23 16:49:53,768::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,768::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,768::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,768::__init__::232::IOProcess::(_processLogs) (50656) Got request for method 'rename' >Thread-41::DEBUG::2014-10-23 16:49:53,769::__init__::232::IOProcess::(_processLogs) (50656) Queuing response >Thread-49098::DEBUG::2014-10-23 16:49:53,769::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-49098::WARNING::2014-10-23 16:49:53,769::sp::927::Storage.StoragePool::(forcedDetachSD) Force detaching domain `748c2d80-0413-4285-ae64-fade24166b4b` >Thread-49098::INFO::2014-10-23 16:49:53,769::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49098::DEBUG::2014-10-23 16:49:53,769::sp::1143::Storage.StoragePool::(_cleanupDomainLinks) Removing: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/748c2d80-0413-4285-ae64-fade24166b4b >Thread-49098::DEBUG::2014-10-23 16:49:53,769::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49098::DEBUG::2014-10-23 16:49:53,770::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49098::DEBUG::2014-10-23 16:49:53,770::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49098::DEBUG::2014-10-23 16:49:53,770::sp::944::Storage.StoragePool::(forcedDetachSD) Force detach for domain `748c2d80-0413-4285-ae64-fade24166b4b` is done >Thread-49098::INFO::2014-10-23 16:49:53,770::logUtils::47::dispatcher::(wrapper) Run and protect: detachStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:49:53,770::task::1191::Storage.TaskManager.Task::(prepare) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::finished: None >Thread-49098::DEBUG::2014-10-23 16:49:53,770::task::595::Storage.TaskManager.Task::(_updateState) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:53,770::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.748c2d80-0413-4285-ae64-fade24166b4b': < ResourceRef 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:53,770::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:53,770::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' >Thread-49098::DEBUG::2014-10-23 16:49:53,770::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:53,771::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:53,771::task::993::Storage.TaskManager.Task::(_decref) Task=`6b1e2a0a-1936-4af5-a5ff-f952849f65c1`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:53,781::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49192::DEBUG::2014-10-23 16:49:53,782::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5877dc6-bfb0-4327-a1d5-0031c142ad3b`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:53,782::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active'}, options=None) >Thread-49192::DEBUG::2014-10-23 16:49:53,782::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`ade68851-a882-48fa-a863-128a306107db`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:49:53,782::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:49:53,782::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:49:53,782::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`ade68851-a882-48fa-a863-128a306107db`::Granted request >Thread-49192::DEBUG::2014-10-23 16:49:53,783::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`39266fd2-83c5-4b55-97cd-fcd8e1332033`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:49:53,783::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49192::DEBUG::2014-10-23 16:49:53,783::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:49:53,783::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`39266fd2-83c5-4b55-97cd-fcd8e1332033`::Granted request >Thread-49192::DEBUG::2014-10-23 16:49:53,783::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active'} >Thread-49192::INFO::2014-10-23 16:49:53,783::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49192::DEBUG::2014-10-23 16:49:53,783::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,784::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:53,784::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:53,784::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:53,784::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:53,784::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:53,785::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:53,794::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:53,795::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:53,849::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,850::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,851::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:53,852::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) (50657) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) (50658) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,853::__init__::232::IOProcess::(_processLogs) (50657) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:53,854::__init__::232::IOProcess::(_processLogs) (50658) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,857::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:49:53,857::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,857::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,857::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,857::__init__::232::IOProcess::(_processLogs) (50659) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:53,876::__init__::232::IOProcess::(_processLogs) (50659) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,877::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:53,877::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,877::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,877::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,877::__init__::232::IOProcess::(_processLogs) (50660) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:53,877::__init__::232::IOProcess::(_processLogs) (50660) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,878::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:49:53,878::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,878::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,878::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,878::__init__::232::IOProcess::(_processLogs) (50661) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:53,878::__init__::232::IOProcess::(_processLogs) (50661) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:53,879::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,879::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,879::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,879::__init__::232::IOProcess::(_processLogs) (50662) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,879::__init__::232::IOProcess::(_processLogs) (50662) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:53,880::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49192::INFO::2014-10-23 16:49:53,880::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49192::INFO::2014-10-23 16:49:53,880::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49192::DEBUG::2014-10-23 16:49:53,880::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-49192::DEBUG::2014-10-23 16:49:53,880::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49192::DEBUG::2014-10-23 16:49:53,880::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:53,880::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:53,880::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49192::WARNING::2014-10-23 16:49:53,880::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe already exists >Thread-49192::DEBUG::2014-10-23 16:49:53,881::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:53,881::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:53,909::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:53,954::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125868 s, 81.4 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:53,990::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:53,991::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:49:53,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,993::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,993::__init__::232::IOProcess::(_processLogs) (50663) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,993::__init__::232::IOProcess::(_processLogs) (50663) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:53,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:53,993::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:53,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:53,994::__init__::232::IOProcess::(_processLogs) (50664) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:53,994::__init__::232::IOProcess::(_processLogs) (50664) Queuing response >Thread-49192::INFO::2014-10-23 16:49:53,995::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:49:53,995::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::INFO::2014-10-23 16:49:53,995::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:49:53,995::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:49:53,995::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:49:53,995::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:49:53,995::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:49:53,995::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49192::INFO::2014-10-23 16:49:53,996::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49192::DEBUG::2014-10-23 16:49:53,996::task::1191::Storage.TaskManager.Task::(prepare) Task=`a5877dc6-bfb0-4327-a1d5-0031c142ad3b`::finished: True >Thread-49192::DEBUG::2014-10-23 16:49:53,996::task::595::Storage.TaskManager.Task::(_updateState) Task=`a5877dc6-bfb0-4327-a1d5-0031c142ad3b`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:53,996::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:53,996::task::993::Storage.TaskManager.Task::(_decref) Task=`a5877dc6-bfb0-4327-a1d5-0031c142ad3b`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:54,007::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [34e164a0] >Thread-49192::DEBUG::2014-10-23 16:49:54,007::task::595::Storage.TaskManager.Task::(_updateState) Task=`ade59ae2-1fc1-4b92-9a4a-01ddb8553e21`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:54,008::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a811e94-b897-42aa-bdf1-3067374bd758', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:49:54,008::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg01 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:54,041::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:54,041::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:54,042::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:54,042::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:54,042::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:54,042::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:54,052::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:54,053::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:54,149::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:54,149::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,149::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,149::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,149::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,149::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,150::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:54,150::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:49:54,150::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:54,150::task::1191::Storage.TaskManager.Task::(prepare) Task=`ade59ae2-1fc1-4b92-9a4a-01ddb8553e21`::finished: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:54,150::task::595::Storage.TaskManager.Task::(_updateState) Task=`ade59ae2-1fc1-4b92-9a4a-01ddb8553e21`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:54,150::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:54,150::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:54,150::task::993::Storage.TaskManager.Task::(_decref) Task=`ade59ae2-1fc1-4b92-9a4a-01ddb8553e21`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:55,963::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:56,004::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117117 s, 87.4 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:57,222::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:57,222::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:57,223::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:57,223::__init__::232::IOProcess::(_processLogs) (50665) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:57,223::__init__::232::IOProcess::(_processLogs) (50665) Queuing response >Thread-49080::DEBUG::2014-10-23 16:49:57,223::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49080::DEBUG::2014-10-23 16:49:57,229::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000327397 s, 980 kB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:49:57,230::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:57,230::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:57,230::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:57,231::__init__::232::IOProcess::(_processLogs) (50666) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:49:57,231::__init__::232::IOProcess::(_processLogs) (50666) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:57,231::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) (50667) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) (50667) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:57,232::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) (50668) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) (50668) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) (50669) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:57,233::__init__::232::IOProcess::(_processLogs) (50669) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:57,473::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49192::DEBUG::2014-10-23 16:49:57,474::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba0ef0da-9817-4a47-b7c5-9676871791fa`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:57,474::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::INFO::2014-10-23 16:49:57,486::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:57,486::task::1191::Storage.TaskManager.Task::(prepare) Task=`ba0ef0da-9817-4a47-b7c5-9676871791fa`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-49192::DEBUG::2014-10-23 16:49:57,486::task::595::Storage.TaskManager.Task::(_updateState) Task=`ba0ef0da-9817-4a47-b7c5-9676871791fa`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:57,486::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:57,486::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:57,486::task::993::Storage.TaskManager.Task::(_decref) Task=`ba0ef0da-9817-4a47-b7c5-9676871791fa`::ref 0 aborting False >Thread-49098::DEBUG::2014-10-23 16:49:57,496::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] >Thread-49098::DEBUG::2014-10-23 16:49:57,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:49:57,496::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49098::DEBUG::2014-10-23 16:49:57,496::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`c1fce36b-b819-4c21-b7eb-adb30dc44b2e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-49098::DEBUG::2014-10-23 16:49:57,497::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'shared' >Thread-49098::DEBUG::2014-10-23 16:49:57,497::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'shared' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:49:57,497::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`c1fce36b-b819-4c21-b7eb-adb30dc44b2e`::Granted request >Thread-49098::DEBUG::2014-10-23 16:49:57,497::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (shared) >Thread-49098::DEBUG::2014-10-23 16:49:57,497::task::993::Storage.TaskManager.Task::(_decref) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:49:57,510::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:57,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 3L, 'domains': '8851d365-d6eb-4bd4-b817-e27df4b7becb:Active', 'master_uuid': u'8851d365-d6eb-4bd4-b817-e27df4b7becb', 'version': '3', 'spm_id': 1, 'type': 'NFS', 'master_ver': 2}, 'dominfo': {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'status': 'Active', 'diskfree': '140340920320', 'isoprefix': '', 'alerts': [], 'disktotal': '143325954048', 'version': 3}}} >Thread-49098::DEBUG::2014-10-23 16:49:57,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:49:57,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:49:57,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:49:57,511::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:49:57,511::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:49:57,511::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:49:57,511::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:49:57,511::task::993::Storage.TaskManager.Task::(_decref) Task=`4778a534-b4c4-4a67-add6-d0a37d101658`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:49:58,013::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:49:58,053::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0118838 s, 86.2 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:58,528::task::595::Storage.TaskManager.Task::(_updateState) Task=`52a7ad45-a6bc-4eb3-b9d2-8c72f978a011`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:58,528::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:49:58,528::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000327397', 'lastCheck': '1.3', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:58,529::task::1191::Storage.TaskManager.Task::(prepare) Task=`52a7ad45-a6bc-4eb3-b9d2-8c72f978a011`::finished: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000327397', 'lastCheck': '1.3', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:49:58,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`52a7ad45-a6bc-4eb3-b9d2-8c72f978a011`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:58,529::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:58,529::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:58,529::task::993::Storage.TaskManager.Task::(_decref) Task=`52a7ad45-a6bc-4eb3-b9d2-8c72f978a011`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:59,256::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [28b8ce37] >Thread-49192::DEBUG::2014-10-23 16:49:59,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb19cf9-9528-4a57-b6bd-8672111171a6`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:59,256::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a811e94-b897-42aa-bdf1-3067374bd758', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:49:59,257::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/10.34.63.199:_cmg01 >Thread-49192::DEBUG::2014-10-23 16:49:59,257::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.34.63.199:/cmg01 /rhev/data-center/mnt/10.34.63.199:_cmg01 (cwd None) >Thread-41::DEBUG::2014-10-23 16:49:59,276::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,276::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,277::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,277::__init__::232::IOProcess::(_processLogs) (50670) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:59,277::__init__::232::IOProcess::(_processLogs) (50670) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,285::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.199:_cmg01 >Thread-41::DEBUG::2014-10-23 16:49:59,285::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,285::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,285::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,285::__init__::232::IOProcess::(_processLogs) (50671) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,286::__init__::232::IOProcess::(_processLogs) (50671) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,286::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'748c2d80-0413-4285-ae64-fade24166b4b',) >Thread-49192::DEBUG::2014-10-23 16:49:59,286::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49192::INFO::2014-10-23 16:49:59,286::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:59,286::task::1191::Storage.TaskManager.Task::(prepare) Task=`1eb19cf9-9528-4a57-b6bd-8672111171a6`::finished: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:59,286::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb19cf9-9528-4a57-b6bd-8672111171a6`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:59,286::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:59,287::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:59,287::task::993::Storage.TaskManager.Task::(_decref) Task=`1eb19cf9-9528-4a57-b6bd-8672111171a6`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:59,293::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [28b8ce37] >Thread-49192::DEBUG::2014-10-23 16:49:59,293::task::595::Storage.TaskManager.Task::(_updateState) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:59,293::logUtils::44::dispatcher::(wrapper) Run and protect: formatStorageDomain(sdUUID='748c2d80-0413-4285-ae64-fade24166b4b', autoDetach=False, options=None) >Thread-49192::DEBUG::2014-10-23 16:49:59,294::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:59,294::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:59,294::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:59,294::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,304::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:59,305::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,399::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:59,400::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`42a0b52c-2934-4194-9ba9-916488a9477c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2747' at 'formatStorageDomain' >Thread-49192::DEBUG::2014-10-23 16:49:59,400::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:49:59,400::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:49:59,400::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.748c2d80-0413-4285-ae64-fade24166b4b`ReqID=`42a0b52c-2934-4194-9ba9-916488a9477c`::Granted request >Thread-49192::DEBUG::2014-10-23 16:49:59,400::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::_resourcesAcquired: Storage.748c2d80-0413-4285-ae64-fade24166b4b (exclusive) >Thread-49192::DEBUG::2014-10-23 16:49:59,401::task::993::Storage.TaskManager.Task::(_decref) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::ref 1 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:59,401::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:59,401::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:59,401::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:59,401::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:59,401::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:59,401::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,412::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:59,412::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,466::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:59,466::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,466::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,466::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,466::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,466::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,467::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,467::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:49:59,468::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,469::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,469::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,469::__init__::232::IOProcess::(_processLogs) (50672) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,469::__init__::232::IOProcess::(_processLogs) (50672) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,469::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,470::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,470::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,470::__init__::232::IOProcess::(_processLogs) (50673) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,470::__init__::232::IOProcess::(_processLogs) (50673) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,472::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg01/748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:49:59,472::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,472::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,472::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,472::__init__::232::IOProcess::(_processLogs) (50674) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:49:59,473::__init__::232::IOProcess::(_processLogs) (50674) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,473::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:49:59,474::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,474::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,474::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,474::__init__::232::IOProcess::(_processLogs) (50675) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:59,475::__init__::232::IOProcess::(_processLogs) (50675) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,475::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f1dd9f262ee1d606554abe25e457a9443edbd96f'] >Thread-41::DEBUG::2014-10-23 16:49:59,475::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,475::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) (50676) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) (50676) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,476::__init__::232::IOProcess::(_processLogs) (50677) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,477::__init__::232::IOProcess::(_processLogs) (50677) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,477::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49192::INFO::2014-10-23 16:49:59,477::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_imageNS already registered >Thread-49192::INFO::2014-10-23 16:49:59,477::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 748c2d80-0413-4285-ae64-fade24166b4b_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:49:59,478::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,478::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,478::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,478::__init__::232::IOProcess::(_processLogs) (50678) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:49:59,478::__init__::232::IOProcess::(_processLogs) (50678) Queuing response >Thread-49192::DEBUG::2014-10-23 16:49:59,478::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'MASTER_VERSION=1', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=748c2d80-0413-4285-ae64-fade24166b4b:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg01', 'ROLE=Regular', 'SDUUID=748c2d80-0413-4285-ae64-fade24166b4b', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=f1dd9f262ee1d606554abe25e457a9443edbd96f'] >Thread-49192::INFO::2014-10-23 16:49:59,478::fileSD::510::Storage.StorageDomain::(format) Formatting domain 748c2d80-0413-4285-ae64-fade24166b4b >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) (50679) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) (50679) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,480::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,481::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,481::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,481::__init__::232::IOProcess::(_processLogs) (50680) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:49:59,481::__init__::232::IOProcess::(_processLogs) (50680) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,483::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,483::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,483::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,483::__init__::232::IOProcess::(_processLogs) (50681) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,483::__init__::232::IOProcess::(_processLogs) (50681) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,484::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,484::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,484::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,484::__init__::232::IOProcess::(_processLogs) (50682) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,484::__init__::232::IOProcess::(_processLogs) (50682) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) (50683) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) (50683) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,485::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,486::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,486::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,486::__init__::232::IOProcess::(_processLogs) (50684) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,486::__init__::232::IOProcess::(_processLogs) (50684) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,486::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,487::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,487::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,487::__init__::232::IOProcess::(_processLogs) (50685) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,487::__init__::232::IOProcess::(_processLogs) (50685) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,488::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,488::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,488::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,488::__init__::232::IOProcess::(_processLogs) (50686) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,488::__init__::232::IOProcess::(_processLogs) (50686) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,489::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,489::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,489::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,489::__init__::232::IOProcess::(_processLogs) (50687) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,489::__init__::232::IOProcess::(_processLogs) (50687) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,490::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,490::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,490::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,490::__init__::232::IOProcess::(_processLogs) (50688) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,490::__init__::232::IOProcess::(_processLogs) (50688) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,491::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,491::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,491::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,491::__init__::232::IOProcess::(_processLogs) (50689) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,492::__init__::232::IOProcess::(_processLogs) (50689) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,492::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,492::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,492::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,492::__init__::232::IOProcess::(_processLogs) (50690) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,493::__init__::232::IOProcess::(_processLogs) (50690) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,493::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,493::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,493::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,493::__init__::232::IOProcess::(_processLogs) (50691) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,494::__init__::232::IOProcess::(_processLogs) (50691) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,494::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,495::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,495::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,495::__init__::232::IOProcess::(_processLogs) (50692) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,495::__init__::232::IOProcess::(_processLogs) (50692) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,495::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,496::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,496::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,496::__init__::232::IOProcess::(_processLogs) (50693) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,496::__init__::232::IOProcess::(_processLogs) (50693) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,497::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,497::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,497::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,497::__init__::232::IOProcess::(_processLogs) (50694) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,497::__init__::232::IOProcess::(_processLogs) (50694) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,498::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,498::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,498::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,498::__init__::232::IOProcess::(_processLogs) (50695) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,498::__init__::232::IOProcess::(_processLogs) (50695) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,499::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,499::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,499::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,499::__init__::232::IOProcess::(_processLogs) (50696) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,499::__init__::232::IOProcess::(_processLogs) (50696) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,500::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,500::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,500::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,500::__init__::232::IOProcess::(_processLogs) (50697) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,500::__init__::232::IOProcess::(_processLogs) (50697) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,501::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,501::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,501::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,501::__init__::232::IOProcess::(_processLogs) (50698) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,501::__init__::232::IOProcess::(_processLogs) (50698) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,502::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,502::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,502::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,502::__init__::232::IOProcess::(_processLogs) (50699) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,502::__init__::232::IOProcess::(_processLogs) (50699) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,503::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,503::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,503::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,503::__init__::232::IOProcess::(_processLogs) (50700) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) (50700) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) (50701) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,504::__init__::232::IOProcess::(_processLogs) (50701) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,505::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,505::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,505::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,505::__init__::232::IOProcess::(_processLogs) (50702) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,507::__init__::232::IOProcess::(_processLogs) (50702) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,507::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,507::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,507::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) (50703) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) (50703) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,508::__init__::232::IOProcess::(_processLogs) (50704) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,509::__init__::232::IOProcess::(_processLogs) (50704) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,509::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,509::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,510::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,510::__init__::232::IOProcess::(_processLogs) (50705) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,510::__init__::232::IOProcess::(_processLogs) (50705) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,511::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,511::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,511::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,511::__init__::232::IOProcess::(_processLogs) (50706) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,513::__init__::232::IOProcess::(_processLogs) (50706) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,513::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,514::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,514::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,514::__init__::232::IOProcess::(_processLogs) (50707) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,516::__init__::232::IOProcess::(_processLogs) (50707) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,517::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,517::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,517::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,517::__init__::232::IOProcess::(_processLogs) (50708) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,517::__init__::232::IOProcess::(_processLogs) (50708) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,518::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,518::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,518::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,518::__init__::232::IOProcess::(_processLogs) (50709) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,519::__init__::232::IOProcess::(_processLogs) (50709) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,520::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,520::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,520::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,520::__init__::232::IOProcess::(_processLogs) (50710) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,520::__init__::232::IOProcess::(_processLogs) (50710) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,521::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,521::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,521::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,521::__init__::232::IOProcess::(_processLogs) (50711) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,522::__init__::232::IOProcess::(_processLogs) (50711) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,522::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,522::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,522::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,522::__init__::232::IOProcess::(_processLogs) (50712) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,523::__init__::232::IOProcess::(_processLogs) (50712) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,523::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,523::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,523::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,523::__init__::232::IOProcess::(_processLogs) (50713) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:49:59,524::__init__::232::IOProcess::(_processLogs) (50713) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,524::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,524::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,525::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,525::__init__::232::IOProcess::(_processLogs) (50714) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,525::__init__::232::IOProcess::(_processLogs) (50714) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,525::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,525::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,526::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,526::__init__::232::IOProcess::(_processLogs) (50715) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,526::__init__::232::IOProcess::(_processLogs) (50715) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,526::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,526::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) (50716) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) (50716) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,527::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,528::__init__::232::IOProcess::(_processLogs) (50717) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:49:59,528::__init__::232::IOProcess::(_processLogs) (50717) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,528::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,528::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) (50718) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) (50718) Queuing response >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:49:59,529::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:49:59,530::__init__::232::IOProcess::(_processLogs) (50719) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:49:59,531::__init__::232::IOProcess::(_processLogs) (50719) Queuing response >Thread-49192::INFO::2014-10-23 16:49:59,531::logUtils::47::dispatcher::(wrapper) Run and protect: formatStorageDomain, Return response: None >Thread-49192::DEBUG::2014-10-23 16:49:59,531::task::1191::Storage.TaskManager.Task::(prepare) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::finished: None >Thread-49192::DEBUG::2014-10-23 16:49:59,531::task::595::Storage.TaskManager.Task::(_updateState) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:59,531::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.748c2d80-0413-4285-ae64-fade24166b4b': < ResourceRef 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', isValid: 'True' obj: 'None'>} >Thread-49192::DEBUG::2014-10-23 16:49:59,532::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:59,532::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' >Thread-49192::DEBUG::2014-10-23 16:49:59,532::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:49:59,532::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:49:59,532::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.748c2d80-0413-4285-ae64-fade24166b4b', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:49:59,532::task::993::Storage.TaskManager.Task::(_decref) Task=`d332460e-15d6-438b-a690-dfc089e45d4e`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:49:59,583::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [72a28b83] >Thread-49192::DEBUG::2014-10-23 16:49:59,584::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4424cee-d8bb-437e-be86-217ee23b9c99`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:49:59,584::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg01', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '3a811e94-b897-42aa-bdf1-3067374bd758', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:49:59,584::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg01 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,608::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:49:59,608::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:59,608::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:49:59,608::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:49:59,608::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:49:59,608::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,618::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:49:59,619::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:49:59,691::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:49:59,691::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,691::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,691::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,691::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,691::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,692::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:49:59,692::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:49:59,692::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:59,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`a4424cee-d8bb-437e-be86-217ee23b9c99`::finished: {'statuslist': [{'status': 0, 'id': '3a811e94-b897-42aa-bdf1-3067374bd758'}]} >Thread-49192::DEBUG::2014-10-23 16:49:59,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`a4424cee-d8bb-437e-be86-217ee23b9c99`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:49:59,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:49:59,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:49:59,692::task::993::Storage.TaskManager.Task::(_decref) Task=`a4424cee-d8bb-437e-be86-217ee23b9c99`::ref 0 aborting False >Dummy-2584::DEBUG::2014-10-23 16:50:00,063::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:50:00,103::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0117688 s, 87.0 MB/s\n'; <rc> = 0 >Dummy-2584::DEBUG::2014-10-23 16:50:02,112::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-2584::DEBUG::2014-10-23 16:50:02,155::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116861 s, 87.6 MB/s\n'; <rc> = 0 >Thread-49098::DEBUG::2014-10-23 16:50:03,325::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [61b99b3b] >Thread-49098::DEBUG::2014-10-23 16:50:03,330::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::moving from state init -> state preparing >Thread-49098::INFO::2014-10-23 16:50:03,331::logUtils::44::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', msdUUID='00000000-0000-0000-0000-000000000000', masterVersion=2, options=None) >Thread-49098::DEBUG::2014-10-23 16:50:03,331::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`0bb883ba-2af0-4edf-b8e6-0543c016092a`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1227' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:50:03,331::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:50:03,331::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:50:03,331::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`0bb883ba-2af0-4edf-b8e6-0543c016092a`::Granted request >Thread-49098::DEBUG::2014-10-23 16:50:03,332::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49098::DEBUG::2014-10-23 16:50:03,332::task::993::Storage.TaskManager.Task::(_decref) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::ref 1 aborting False >Thread-49098::DEBUG::2014-10-23 16:50:03,332::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`5af1c200-de91-400c-af02-3d3c216ba5c8`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1228' at 'deactivateStorageDomain' >Thread-49098::DEBUG::2014-10-23 16:50:03,332::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' for lock type 'exclusive' >Thread-49098::DEBUG::2014-10-23 16:50:03,332::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free. Now locking as 'exclusive' (1 active user) >Thread-49098::DEBUG::2014-10-23 16:50:03,332::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`5af1c200-de91-400c-af02-3d3c216ba5c8`::Granted request >Thread-49098::DEBUG::2014-10-23 16:50:03,333::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::_resourcesAcquired: Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb (exclusive) >Thread-49098::DEBUG::2014-10-23 16:50:03,333::task::993::Storage.TaskManager.Task::(_decref) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::ref 1 aborting False >Thread-49098::INFO::2014-10-23 16:50:03,334::sp::1070::Storage.StoragePool::(deactivateSD) sdUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-49098::INFO::2014-10-23 16:50:03,334::sp::1094::Storage.StoragePool::(deactivateSD) Silently ignoring the deactivation request for the master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49098::INFO::2014-10-23 16:50:03,334::logUtils::47::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-49098::DEBUG::2014-10-23 16:50:03,334::task::1191::Storage.TaskManager.Task::(prepare) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::finished: None >Thread-49098::DEBUG::2014-10-23 16:50:03,334::task::595::Storage.TaskManager.Task::(_updateState) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::moving from state preparing -> state finished >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb': < ResourceRef 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:50:03,334::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:50:03,335::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49098::DEBUG::2014-10-23 16:50:03,335::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49098::DEBUG::2014-10-23 16:50:03,335::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49098::DEBUG::2014-10-23 16:50:03,335::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49098::DEBUG::2014-10-23 16:50:03,335::task::993::Storage.TaskManager.Task::(_decref) Task=`fd98fdeb-8133-4837-b914-1871ae9bbd5a`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:03,365::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [61b99b3b] >Thread-49192::DEBUG::2014-10-23 16:50:03,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dcb1f7c-86c5-4e9a-87ce-a14b78369698`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:03,365::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:50:03,366::taskManager::112::Storage.TaskManager::(getAllTasksStatuses) Entry. >Thread-49192::DEBUG::2014-10-23 16:50:03,366::taskManager::123::Storage.TaskManager::(getAllTasksStatuses) Return: {} >Thread-49192::INFO::2014-10-23 16:50:03,366::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-49192::DEBUG::2014-10-23 16:50:03,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`9dcb1f7c-86c5-4e9a-87ce-a14b78369698`::finished: {'allTasksStatus': {}} >Thread-49192::DEBUG::2014-10-23 16:50:03,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`9dcb1f7c-86c5-4e9a-87ce-a14b78369698`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:03,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:03,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:03,366::task::993::Storage.TaskManager.Task::(_decref) Task=`9dcb1f7c-86c5-4e9a-87ce-a14b78369698`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:03,384::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [61b99b3b] >Thread-49192::DEBUG::2014-10-23 16:50:03,385::task::595::Storage.TaskManager.Task::(_updateState) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:03,385::logUtils::44::dispatcher::(wrapper) Run and protect: spmStop(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::DEBUG::2014-10-23 16:50:03,385::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`930f05eb-948c-4596-9e1a-7f9c81f25de2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '596' at 'spmStop' >Thread-49192::DEBUG::2014-10-23 16:50:03,385::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:50:03,385::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:50:03,385::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`930f05eb-948c-4596-9e1a-7f9c81f25de2`::Granted request >Thread-49192::DEBUG::2014-10-23 16:50:03,385::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49192::DEBUG::2014-10-23 16:50:03,385::task::993::Storage.TaskManager.Task::(_decref) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::ref 1 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:03,386::sp::337::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9bc34ec0-35c0-452a-8fbe-6d5704674ff8`::Request was made in '/usr/share/vdsm/storage/sp.py' line '339' at '_shutDownUpgrade' >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`9bc34ec0-35c0-452a-8fbe-6d5704674ff8`::Granted request >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:50:03,386::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::INFO::2014-10-23 16:50:03,387::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:50:03,389::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb successfully released >Thread-49192::INFO::2014-10-23 16:50:03,389::logUtils::47::dispatcher::(wrapper) Run and protect: spmStop, Return response: None >Thread-49192::DEBUG::2014-10-23 16:50:03,389::task::1191::Storage.TaskManager.Task::(prepare) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::finished: None >Thread-49192::DEBUG::2014-10-23 16:50:03,389::task::595::Storage.TaskManager.Task::(_updateState) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:03,389::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49192::DEBUG::2014-10-23 16:50:03,389::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:03,389::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:50:03,390::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:50:03,390::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:50:03,390::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:50:03,390::task::993::Storage.TaskManager.Task::(_decref) Task=`c35358af-07d9-4aa6-99cd-e32c4094749c`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:03,466::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [61b99b3b] >Thread-49192::DEBUG::2014-10-23 16:50:03,467::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:03,467::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, remove=False, options=None) >Thread-49192::DEBUG::2014-10-23 16:50:03,467::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`25c7d598-7030-426e-84b5-ef41c4ddb581`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1124' at 'disconnectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:50:03,467::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:50:03,467::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:50:03,467::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`25c7d598-7030-426e-84b5-ef41c4ddb581`::Granted request >Thread-49192::DEBUG::2014-10-23 16:50:03,468::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49192::DEBUG::2014-10-23 16:50:03,468::task::993::Storage.TaskManager.Task::(_decref) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::ref 1 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:03,468::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`785d0c66-a8c7-4bc0-a494-06b8339c1a6d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1132' at '_disconnectPool' >Thread-49192::DEBUG::2014-10-23 16:50:03,468::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:50:03,468::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:50:03,468::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`785d0c66-a8c7-4bc0-a494-06b8339c1a6d`::Granted request >Thread-49192::INFO::2014-10-23 16:50:03,468::sp::657::Storage.StoragePool::(disconnect) Disconnect from the storage pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49090::INFO::2014-10-23 16:50:04,540::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-49090::INFO::2014-10-23 16:50:04,540::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-49090::DEBUG::2014-10-23 16:50:04,540::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-49090::DEBUG::2014-10-23 16:50:04,553::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00644124 s, 636 kB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:50:05,284::fileUtils::124::Storage.fileUtils::(cleanupdir) Removing directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49192::INFO::2014-10-23 16:50:05,284::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49080::DEBUG::2014-10-23 16:50:05,284::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49080::INFO::2014-10-23 16:50:05,285::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb (id: 1) >Dummy-2584::INFO::2014-10-23 16:50:05,286::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-49080::DEBUG::2014-10-23 16:50:07,286::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb released successfully (id: 1) >Thread-49192::DEBUG::2014-10-23 16:50:07,287::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49192::DEBUG::2014-10-23 16:50:07,287::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:50:07,287::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:50:07,287::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49192::INFO::2014-10-23 16:50:07,287::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True >Thread-49192::DEBUG::2014-10-23 16:50:07,287::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::finished: True >Thread-49192::DEBUG::2014-10-23 16:50:07,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:07,287::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49192::DEBUG::2014-10-23 16:50:07,288::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:07,288::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:50:07,288::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:50:07,288::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:50:07,288::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:50:07,288::task::993::Storage.TaskManager.Task::(_decref) Task=`6e2194b8-5db2-40ca-aeba-8030885d846a`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:07,337::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [61b99b3b] >Thread-49192::DEBUG::2014-10-23 16:50:07,337::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb5d7017-a6c6-40a2-966b-1956dd792286`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:07,337::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, remove=False, options=None) >Thread-49192::WARNING::2014-10-23 16:50:07,337::hsm::1119::Storage.HSM::(disconnectStoragePool) disconnect sp: 11cfd6be-5798-4973-9e8b-a3262a2d3ebe failed. Known pools {} >Thread-49192::INFO::2014-10-23 16:50:07,337::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: None >Thread-49192::DEBUG::2014-10-23 16:50:07,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`eb5d7017-a6c6-40a2-966b-1956dd792286`::finished: None >Thread-49192::DEBUG::2014-10-23 16:50:07,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`eb5d7017-a6c6-40a2-966b-1956dd792286`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:07,338::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:07,338::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:07,338::task::993::Storage.TaskManager.Task::(_decref) Task=`eb5d7017-a6c6-40a2-966b-1956dd792286`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:07,420::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [23e82026] >Thread-49192::DEBUG::2014-10-23 16:50:07,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`6004b50a-1c62-4807-acc5-3315c3a758fb`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:07,420::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:50:07,421::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:50:07,465::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:50:07,465::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:50:07,465::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:50:07,465::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:50:07,465::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:50:07,466::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:50:07,476::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:50:07,478::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:50:07,555::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,556::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,557::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:50:07,557::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:50:07,557::task::1191::Storage.TaskManager.Task::(prepare) Task=`6004b50a-1c62-4807-acc5-3315c3a758fb`::finished: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:50:07,557::task::595::Storage.TaskManager.Task::(_updateState) Task=`6004b50a-1c62-4807-acc5-3315c3a758fb`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:07,557::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:07,557::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:07,557::task::993::Storage.TaskManager.Task::(_decref) Task=`6004b50a-1c62-4807-acc5-3315c3a758fb`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:07,567::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [4b02505a] >Thread-49192::DEBUG::2014-10-23 16:50:07,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`08cec0e7-941c-4a6e-b3ca-2de02b177156`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:07,567::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:50:07,568::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-49192::ERROR::2014-10-23 16:50:07,578::hsm::2531::Storage.HSM::(disconnectStorageServer) Could not disconnect from storageServer >Traceback (most recent call last): > File "/usr/share/vdsm/storage/hsm.py", line 2527, in disconnectStorageServer > conObj.disconnect() > File "/usr/share/vdsm/storage/storageServer.py", line 334, in disconnect > return self._mountCon.disconnect() > File "/usr/share/vdsm/storage/storageServer.py", line 235, in disconnect > self._mount.umount(True, True) > File "/usr/share/vdsm/storage/mount.py", line 254, in umount > return self._runcmd(cmd, timeout) > File "/usr/share/vdsm/storage/mount.py", line 239, in _runcmd > raise MountError(rc, ";".join((out, err))) >MountError: (1, ';umount: /rhev/data-center/mnt/10.34.63.199:_cmg03: not found\n') >Thread-49192::DEBUG::2014-10-23 16:50:07,579::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:50:07,579::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:50:07,579::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:50:07,579::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:50:07,580::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:50:07,580::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:50:07,589::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:50:07,590::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:50:07,674::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:50:07,674::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,674::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,675::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,675::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,675::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,675::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:50:07,675::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:50:07,676::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 477, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:50:07,676::task::1191::Storage.TaskManager.Task::(prepare) Task=`08cec0e7-941c-4a6e-b3ca-2de02b177156`::finished: {'statuslist': [{'status': 477, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:50:07,676::task::595::Storage.TaskManager.Task::(_updateState) Task=`08cec0e7-941c-4a6e-b3ca-2de02b177156`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:07,676::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:07,676::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:07,676::task::993::Storage.TaskManager.Task::(_decref) Task=`08cec0e7-941c-4a6e-b3ca-2de02b177156`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:13,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2f94b0c-3f50-4b5d-897c-bcd88a83d0ab`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:13,701::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:50:13,701::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:50:13,701::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2f94b0c-3f50-4b5d-897c-bcd88a83d0ab`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:50:13,701::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2f94b0c-3f50-4b5d-897c-bcd88a83d0ab`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:13,701::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:13,701::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:13,701::task::993::Storage.TaskManager.Task::(_decref) Task=`b2f94b0c-3f50-4b5d-897c-bcd88a83d0ab`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:28,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`35e46d32-197a-4d3e-858f-c9ce3e87a68f`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:28,877::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:50:28,877::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:50:28,877::task::1191::Storage.TaskManager.Task::(prepare) Task=`35e46d32-197a-4d3e-858f-c9ce3e87a68f`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:50:28,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`35e46d32-197a-4d3e-858f-c9ce3e87a68f`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:28,877::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:28,877::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:28,877::task::993::Storage.TaskManager.Task::(_decref) Task=`35e46d32-197a-4d3e-858f-c9ce3e87a68f`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:44,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d38ea8d-24d9-4d83-831e-c98d971c10f9`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:44,051::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:50:44,051::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:50:44,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`1d38ea8d-24d9-4d83-831e-c98d971c10f9`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:50:44,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`1d38ea8d-24d9-4d83-831e-c98d971c10f9`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:44,051::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:44,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:44,051::task::993::Storage.TaskManager.Task::(_decref) Task=`1d38ea8d-24d9-4d83-831e-c98d971c10f9`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:50:59,220::task::595::Storage.TaskManager.Task::(_updateState) Task=`afc179f1-2fea-4ec3-a9be-e07e00280545`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:50:59,221::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:50:59,221::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:50:59,221::task::1191::Storage.TaskManager.Task::(prepare) Task=`afc179f1-2fea-4ec3-a9be-e07e00280545`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:50:59,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`afc179f1-2fea-4ec3-a9be-e07e00280545`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:50:59,221::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:50:59,221::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:50:59,221::task::993::Storage.TaskManager.Task::(_decref) Task=`afc179f1-2fea-4ec3-a9be-e07e00280545`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:51:14,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7aef01-124f-4d82-8684-1faa4e0ae63a`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:51:14,407::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:51:14,407::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:51:14,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`0c7aef01-124f-4d82-8684-1faa4e0ae63a`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:51:14,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`0c7aef01-124f-4d82-8684-1faa4e0ae63a`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:51:14,407::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:51:14,407::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:51:14,407::task::993::Storage.TaskManager.Task::(_decref) Task=`0c7aef01-124f-4d82-8684-1faa4e0ae63a`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:51:29,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`65571ca6-3aa8-4f86-a237-f638b1ac0adb`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:51:29,704::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:51:29,705::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:51:29,705::task::1191::Storage.TaskManager.Task::(prepare) Task=`65571ca6-3aa8-4f86-a237-f638b1ac0adb`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:51:29,705::task::595::Storage.TaskManager.Task::(_updateState) Task=`65571ca6-3aa8-4f86-a237-f638b1ac0adb`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:51:29,705::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:51:29,705::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:51:29,705::task::993::Storage.TaskManager.Task::(_decref) Task=`65571ca6-3aa8-4f86-a237-f638b1ac0adb`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:51:45,024::task::595::Storage.TaskManager.Task::(_updateState) Task=`a146124e-a8fb-49bb-ad41-da53dd183dbe`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:51:45,024::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:51:45,025::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:51:45,025::task::1191::Storage.TaskManager.Task::(prepare) Task=`a146124e-a8fb-49bb-ad41-da53dd183dbe`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:51:45,025::task::595::Storage.TaskManager.Task::(_updateState) Task=`a146124e-a8fb-49bb-ad41-da53dd183dbe`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:51:45,025::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:51:45,025::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:51:45,025::task::993::Storage.TaskManager.Task::(_decref) Task=`a146124e-a8fb-49bb-ad41-da53dd183dbe`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:00,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`d66e0e8b-402b-42b3-8c89-b85da7f2a73f`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:00,365::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:52:00,366::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:52:00,366::task::1191::Storage.TaskManager.Task::(prepare) Task=`d66e0e8b-402b-42b3-8c89-b85da7f2a73f`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:52:00,366::task::595::Storage.TaskManager.Task::(_updateState) Task=`d66e0e8b-402b-42b3-8c89-b85da7f2a73f`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:00,366::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:00,366::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:00,366::task::993::Storage.TaskManager.Task::(_decref) Task=`d66e0e8b-402b-42b3-8c89-b85da7f2a73f`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:05,568::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [23221302] >Thread-49192::DEBUG::2014-10-23 16:52:05,568::task::595::Storage.TaskManager.Task::(_updateState) Task=`074f196d-feeb-4e10-80e0-1e8afab3ac45`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:05,569::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:52:05,570::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/10.34.63.199:_cmg03 >Thread-49192::DEBUG::2014-10-23 16:52:05,570::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.34.63.199:/cmg03 /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-41::DEBUG::2014-10-23 16:52:05,599::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,599::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,599::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,599::__init__::232::IOProcess::(_processLogs) (50720) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:05,600::__init__::232::IOProcess::(_processLogs) (50720) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,612::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.199:_cmg03 >Thread-41::DEBUG::2014-10-23 16:52:05,612::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,613::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,613::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,613::__init__::232::IOProcess::(_processLogs) (50721) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:05,613::__init__::232::IOProcess::(_processLogs) (50721) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,613::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'8851d365-d6eb-4bd4-b817-e27df4b7becb',) >Thread-49192::DEBUG::2014-10-23 16:52:05,614::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49192::INFO::2014-10-23 16:52:05,614::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:05,614::task::1191::Storage.TaskManager.Task::(prepare) Task=`074f196d-feeb-4e10-80e0-1e8afab3ac45`::finished: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:05,614::task::595::Storage.TaskManager.Task::(_updateState) Task=`074f196d-feeb-4e10-80e0-1e8afab3ac45`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:05,614::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:05,614::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:05,614::task::993::Storage.TaskManager.Task::(_decref) Task=`074f196d-feeb-4e10-80e0-1e8afab3ac45`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:05,637::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [23221302] >Thread-49192::DEBUG::2014-10-23 16:52:05,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`f927ed97-d943-40e5-9281-f75892f307a4`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:05,637::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, msdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', masterVersion=2, domainsMap={'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active'}, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`b953a3e4-2bac-4a53-9b06-f0ccb2052278`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`b953a3e4-2bac-4a53-9b06-f0ccb2052278`::Granted request >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`5771adf8-98cf-45b4-9605-1d5e0b6e3abc`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-49192::DEBUG::2014-10-23 16:52:05,638::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:52:05,639::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:52:05,639::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`5771adf8-98cf-45b4-9605-1d5e0b6e3abc`::Granted request >Thread-49192::DEBUG::2014-10-23 16:52:05,639::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 2 and domains map to {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'active'} >Thread-49192::INFO::2014-10-23 16:52:05,639::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {'8851d365-d6eb-4bd4-b817-e27df4b7becb': 'Active'} >Thread-49192::INFO::2014-10-23 16:52:05,639::sp::133::Storage.StoragePool::(setBackend) updating pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe backend from type NoneType instance 0x3533572210 to type StoragePoolMemoryBackend instance 0x7f28d8089ba8 >Thread-49192::INFO::2014-10-23 16:52:05,639::sp::631::Storage.StoragePool::(connect) Connect host #1 to the storage pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe with master domain: 8851d365-d6eb-4bd4-b817-e27df4b7becb (ver = 2) >Thread-49192::DEBUG::2014-10-23 16:52:05,639::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,639::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,639::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,640::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,640::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,640::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,640::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:52:05,640::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:05,640::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:52:05,640::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:05,640::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:52:05,640::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:05,650::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:05,653::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:05,777::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:05,777::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,777::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,778::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,778::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,778::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,778::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:05,778::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:52:05,779::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,779::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,779::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,779::__init__::232::IOProcess::(_processLogs) (50722) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:05,780::__init__::232::IOProcess::(_processLogs) (50722) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,781::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:52:05,781::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,782::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,782::__init__::232::IOProcess::(_processLogs) (50723) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:52:05,783::__init__::232::IOProcess::(_processLogs) (50723) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,783::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:52:05,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,783::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,784::__init__::232::IOProcess::(_processLogs) (50724) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:52:05,785::__init__::232::IOProcess::(_processLogs) (50724) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,785::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:52:05,786::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,786::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,786::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,786::__init__::232::IOProcess::(_processLogs) (50725) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:05,786::__init__::232::IOProcess::(_processLogs) (50725) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:05,787::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,787::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,787::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,787::__init__::232::IOProcess::(_processLogs) (50726) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:05,787::__init__::232::IOProcess::(_processLogs) (50726) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,788::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-49192::INFO::2014-10-23 16:52:05,788::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49192::INFO::2014-10-23 16:52:05,788::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-49192::DEBUG::2014-10-23 16:52:05,788::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb verified, version 2 >Thread-49192::DEBUG::2014-10-23 16:52:05,788::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49192::DEBUG::2014-10-23 16:52:05,788::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::INFO::2014-10-23 16:52:05,788::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:52:05,789::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `11cfd6be-5798-4973-9e8b-a3262a2d3ebe` started monitoring domain `8851d365-d6eb-4bd4-b817-e27df4b7becb` >Thread-49284::DEBUG::2014-10-23 16:52:05,789::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:52:05,789::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:05,789::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-41::DEBUG::2014-10-23 16:52:05,790::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,791::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-49192::DEBUG::2014-10-23 16:52:05,791::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-41::DEBUG::2014-10-23 16:52:05,792::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,793::__init__::232::IOProcess::(_processLogs) (50727) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:52:05,793::__init__::232::IOProcess::(_processLogs) (50727) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:05,795::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-49284::DEBUG::2014-10-23 16:52:05,795::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49284::DEBUG::2014-10-23 16:52:05,805::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.000274168 s, 1.2 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:52:05,806::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,806::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,806::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,806::__init__::232::IOProcess::(_processLogs) (50728) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:52:05,807::__init__::232::IOProcess::(_processLogs) (50728) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:05,807::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) (50729) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) (50729) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,808::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) (50730) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) (50730) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,809::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,810::__init__::232::IOProcess::(_processLogs) (50731) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:05,810::__init__::232::IOProcess::(_processLogs) (50731) Queuing response >Thread-49284::DEBUG::2014-10-23 16:52:05,813::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 8851d365-d6eb-4bd4-b817-e27df4b7becb changed its status to Valid >Thread-49285::DEBUG::2014-10-23 16:52:05,814::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-49284::INFO::2014-10-23 16:52:05,814::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb (id: 1) >Thread-49285::DEBUG::2014-10-23 16:52:05,815::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-49284::DEBUG::2014-10-23 16:52:05,816::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb successfully acquired (id: 1) >Thread-49285::DEBUG::2014-10-23 16:52:05,816::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-49286::INFO::2014-10-23 16:52:05,823::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-49192::DEBUG::2014-10-23 16:52:05,908::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:05,909::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-41::DEBUG::2014-10-23 16:52:05,910::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:05,911::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:05,911::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:05,911::__init__::232::IOProcess::(_processLogs) (50732) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:05,911::__init__::232::IOProcess::(_processLogs) (50732) Queuing response >Thread-49192::INFO::2014-10-23 16:52:05,912::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:52:05,912::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::INFO::2014-10-23 16:52:05,912::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:52:05,913::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb to /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd >Thread-49192::DEBUG::2014-10-23 16:52:05,914::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-49192::DEBUG::2014-10-23 16:52:05,915::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=8 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:05,924::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00227585 s, 1.8 MB/s\n'; <rc> = 0 >Thread-49192::INFO::2014-10-23 16:52:05,924::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-49192::DEBUG::2014-10-23 16:52:05,924::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:05,936::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00644701 s, 635 kB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:05,937::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49192::DEBUG::2014-10-23 16:52:05,937::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe on master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49192::DEBUG::2014-10-23 16:52:05,937::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:52:05,938::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49192::INFO::2014-10-23 16:52:05,938::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-49192::DEBUG::2014-10-23 16:52:05,939::task::1191::Storage.TaskManager.Task::(prepare) Task=`f927ed97-d943-40e5-9281-f75892f307a4`::finished: True >Thread-49192::DEBUG::2014-10-23 16:52:05,939::task::595::Storage.TaskManager.Task::(_updateState) Task=`f927ed97-d943-40e5-9281-f75892f307a4`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:05,939::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:05,939::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:05,939::task::993::Storage.TaskManager.Task::(_decref) Task=`f927ed97-d943-40e5-9281-f75892f307a4`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:06,567::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:06,567::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfd4a5c8-e32d-418e-8603-b198ee2b0f21`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:06,567::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::INFO::2014-10-23 16:52:06,579::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-49192::DEBUG::2014-10-23 16:52:06,579::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfd4a5c8-e32d-418e-8603-b198ee2b0f21`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-49192::DEBUG::2014-10-23 16:52:06,579::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfd4a5c8-e32d-418e-8603-b198ee2b0f21`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:06,579::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:06,579::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:06,579::task::993::Storage.TaskManager.Task::(_decref) Task=`bfd4a5c8-e32d-418e-8603-b198ee2b0f21`::ref 0 aborting False >Thread-49099::DEBUG::2014-10-23 16:52:06,603::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49099::DEBUG::2014-10-23 16:52:06,604::task::595::Storage.TaskManager.Task::(_updateState) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::moving from state init -> state preparing >Thread-49099::INFO::2014-10-23 16:52:06,604::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', prevID=-1, prevLVER='-1', maxHostID=250, domVersion='3', options=None) >Thread-49099::DEBUG::2014-10-23 16:52:06,604::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`cedf106a-5dfe-4a91-b7bf-f666fb58b609`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '572' at 'spmStart' >Thread-49099::DEBUG::2014-10-23 16:52:06,604::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49099::DEBUG::2014-10-23 16:52:06,604::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49099::DEBUG::2014-10-23 16:52:06,604::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`cedf106a-5dfe-4a91-b7bf-f666fb58b609`::Granted request >Thread-49099::DEBUG::2014-10-23 16:52:06,605::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49099::DEBUG::2014-10-23 16:52:06,605::task::993::Storage.TaskManager.Task::(_decref) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::ref 1 aborting False >Thread-49099::DEBUG::2014-10-23 16:52:06,605::taskManager::78::Storage.TaskManager::(scheduleJob) scheduled job spmStart for task 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49099::INFO::2014-10-23 16:52:06,605::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-49099::DEBUG::2014-10-23 16:52:06,605::task::1183::Storage.TaskManager.Task::(prepare) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::Prepare: 1 jobs exist, move to acquiring >Thread-49099::DEBUG::2014-10-23 16:52:06,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::moving from state preparing -> state acquiring >Thread-49099::DEBUG::2014-10-23 16:52:06,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::moving from state acquiring -> state queued >Thread-49099::DEBUG::2014-10-23 16:52:06,605::taskManager::56::Storage.TaskManager::(_queueTask) queuing task: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49099::DEBUG::2014-10-23 16:52:06,605::taskManager::64::Storage.TaskManager::(_queueTask) task queued: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-10::DEBUG::2014-10-23 16:52:06,605::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-49099::DEBUG::2014-10-23 16:52:06,606::task::1188::Storage.TaskManager.Task::(prepare) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::returning >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,606::threadPool::200::Storage.ThreadPool.WorkerThread::(_processNextTask) Task: 34d66b4e-9038-4627-ac50-4802a327481c running: <bound method Task.commit of <storage.task.Task instance at 0x7f28f804dbd8>> with: None >Thread-49099::DEBUG::2014-10-23 16:52:06,606::task::993::Storage.TaskManager.Task::(_decref) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::ref 0 aborting False >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,606::task::1200::Storage.TaskManager.Task::(commit) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::committing task: 34d66b4e-9038-4627-ac50-4802a327481c >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,607::task::595::Storage.TaskManager.Task::(_updateState) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::moving from state queued -> state running >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,608::task::905::Storage.TaskManager.Task::(_runJobs) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f29000b2190>> (args: (-1, '-1', 250, 3) kwargs: {}) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,608::task::331::Storage.TaskManager.Task::(run) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7f29000b2190>> (args: (-1, '-1', 250, 3) kwargs: {}) callback None >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,608::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,608::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:06,608::misc::751::Storage.SamplingMethod::(__call__) Returning last result >34d66b4e-9038-4627-ac50-4802a327481c::INFO::2014-10-23 16:52:06,619::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb (id: 1) >Thread-49192::DEBUG::2014-10-23 16:52:07,612::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:07,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bf9b954-3324-4168-be39-8c08d1b335d5`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:07,612::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:07,612::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:07,612::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:07,612::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:07,612::task::1191::Storage.TaskManager.Task::(prepare) Task=`4bf9b954-3324-4168-be39-8c08d1b335d5`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:07,612::task::595::Storage.TaskManager.Task::(_updateState) Task=`4bf9b954-3324-4168-be39-8c08d1b335d5`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:07,613::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:07,613::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:07,613::task::993::Storage.TaskManager.Task::(_decref) Task=`4bf9b954-3324-4168-be39-8c08d1b335d5`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:08,617::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:08,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`67946951-2afd-410a-8de1-cbf52aacadc0`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:08,617::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:08,617::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:08,617::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:08,618::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:08,618::task::1191::Storage.TaskManager.Task::(prepare) Task=`67946951-2afd-410a-8de1-cbf52aacadc0`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:08,618::task::595::Storage.TaskManager.Task::(_updateState) Task=`67946951-2afd-410a-8de1-cbf52aacadc0`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:08,618::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:08,618::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:08,618::task::993::Storage.TaskManager.Task::(_decref) Task=`67946951-2afd-410a-8de1-cbf52aacadc0`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:09,622::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:09,622::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f8a64b4-2d1d-4b7b-9b90-2dc7079af8e6`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:09,622::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:09,622::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:09,622::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:09,623::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:09,623::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f8a64b4-2d1d-4b7b-9b90-2dc7079af8e6`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:09,623::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f8a64b4-2d1d-4b7b-9b90-2dc7079af8e6`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:09,623::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:09,623::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:09,623::task::993::Storage.TaskManager.Task::(_decref) Task=`6f8a64b4-2d1d-4b7b-9b90-2dc7079af8e6`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:10,627::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:10,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`038485dd-2ca7-4e31-883d-20b24812688b`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:10,627::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:10,627::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:10,627::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:10,627::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:10,627::task::1191::Storage.TaskManager.Task::(prepare) Task=`038485dd-2ca7-4e31-883d-20b24812688b`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:10,627::task::595::Storage.TaskManager.Task::(_updateState) Task=`038485dd-2ca7-4e31-883d-20b24812688b`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:10,627::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:10,628::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:10,628::task::993::Storage.TaskManager.Task::(_decref) Task=`038485dd-2ca7-4e31-883d-20b24812688b`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:11,631::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:11,632::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6e04c2c-61e3-47ca-b532-cc9219e74372`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:11,632::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:11,632::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:11,632::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:11,632::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:11,632::task::1191::Storage.TaskManager.Task::(prepare) Task=`f6e04c2c-61e3-47ca-b532-cc9219e74372`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:11,632::task::595::Storage.TaskManager.Task::(_updateState) Task=`f6e04c2c-61e3-47ca-b532-cc9219e74372`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:11,632::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:11,632::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:11,632::task::993::Storage.TaskManager.Task::(_decref) Task=`f6e04c2c-61e3-47ca-b532-cc9219e74372`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:12,636::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:12,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`65ce3b21-f55e-4af1-9f3d-7afdde2941c4`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:12,636::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:12,636::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:12,637::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:12,637::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:12,637::task::1191::Storage.TaskManager.Task::(prepare) Task=`65ce3b21-f55e-4af1-9f3d-7afdde2941c4`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:12,637::task::595::Storage.TaskManager.Task::(_updateState) Task=`65ce3b21-f55e-4af1-9f3d-7afdde2941c4`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:12,637::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:12,637::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:12,637::task::993::Storage.TaskManager.Task::(_decref) Task=`65ce3b21-f55e-4af1-9f3d-7afdde2941c4`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:13,641::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:13,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`5169051b-50f9-4bfe-b123-b2b66ebc9404`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:13,642::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:13,642::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:13,642::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:13,642::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:13,642::task::1191::Storage.TaskManager.Task::(prepare) Task=`5169051b-50f9-4bfe-b123-b2b66ebc9404`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:13,642::task::595::Storage.TaskManager.Task::(_updateState) Task=`5169051b-50f9-4bfe-b123-b2b66ebc9404`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:13,642::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:13,642::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:13,643::task::993::Storage.TaskManager.Task::(_decref) Task=`5169051b-50f9-4bfe-b123-b2b66ebc9404`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:14,647::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:14,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9e4046d-a99b-49dd-83c7-1f93ecdc8b12`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:14,647::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:14,647::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:14,647::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:14,647::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:14,647::task::1191::Storage.TaskManager.Task::(prepare) Task=`a9e4046d-a99b-49dd-83c7-1f93ecdc8b12`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:14,647::task::595::Storage.TaskManager.Task::(_updateState) Task=`a9e4046d-a99b-49dd-83c7-1f93ecdc8b12`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:14,647::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:14,647::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:14,648::task::993::Storage.TaskManager.Task::(_decref) Task=`a9e4046d-a99b-49dd-83c7-1f93ecdc8b12`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:15,651::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:15,651::task::595::Storage.TaskManager.Task::(_updateState) Task=`b798cd84-9c49-4989-b4e8-16a4b5bc1a6d`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:15,652::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:15,652::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:15,652::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:15,652::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:15,652::task::1191::Storage.TaskManager.Task::(prepare) Task=`b798cd84-9c49-4989-b4e8-16a4b5bc1a6d`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:15,652::task::595::Storage.TaskManager.Task::(_updateState) Task=`b798cd84-9c49-4989-b4e8-16a4b5bc1a6d`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:15,652::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:15,652::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:15,652::task::993::Storage.TaskManager.Task::(_decref) Task=`b798cd84-9c49-4989-b4e8-16a4b5bc1a6d`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:15,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0cb8c29-7163-4b1f-8573-b3349bc20a2e`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:15,718::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:52:15,718::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000274168', 'lastCheck': '9.9', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:52:15,718::task::1191::Storage.TaskManager.Task::(prepare) Task=`d0cb8c29-7163-4b1f-8573-b3349bc20a2e`::finished: {'8851d365-d6eb-4bd4-b817-e27df4b7becb': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000274168', 'lastCheck': '9.9', 'valid': True}} >Thread-49192::DEBUG::2014-10-23 16:52:15,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`d0cb8c29-7163-4b1f-8573-b3349bc20a2e`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:15,719::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:15,719::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:15,719::task::993::Storage.TaskManager.Task::(_decref) Task=`d0cb8c29-7163-4b1f-8573-b3349bc20a2e`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:52:15,822::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:15,823::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:15,823::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:15,823::__init__::232::IOProcess::(_processLogs) (50733) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:52:15,823::__init__::232::IOProcess::(_processLogs) (50733) Queuing response >Thread-49284::DEBUG::2014-10-23 16:52:15,824::fileSD::261::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/dom_md/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-49284::DEBUG::2014-10-23 16:52:15,831::fileSD::261::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '0+1 records in\n0+1 records out\n321 bytes (321 B) copied, 0.00028022 s, 1.1 MB/s\n'; <rc> = 0 >Thread-41::DEBUG::2014-10-23 16:52:15,831::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:15,832::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:15,832::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:15,832::__init__::232::IOProcess::(_processLogs) (50734) Got request for method 'statvfs' >Thread-41::DEBUG::2014-10-23 16:52:15,832::__init__::232::IOProcess::(_processLogs) (50734) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:15,833::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:15,833::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:15,833::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:15,833::__init__::232::IOProcess::(_processLogs) (50735) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) (50735) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) (50736) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:15,834::__init__::232::IOProcess::(_processLogs) (50736) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:15,835::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:15,835::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:15,835::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:15,835::__init__::232::IOProcess::(_processLogs) (50737) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:15,835::__init__::232::IOProcess::(_processLogs) (50737) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:16,656::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:16,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`57160f4d-e50d-4aba-b894-29753ab99b81`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:16,658::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:16,658::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:16,659::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:16,659::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:16,659::task::1191::Storage.TaskManager.Task::(prepare) Task=`57160f4d-e50d-4aba-b894-29753ab99b81`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:16,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`57160f4d-e50d-4aba-b894-29753ab99b81`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:16,659::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:16,659::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:16,659::task::993::Storage.TaskManager.Task::(_decref) Task=`57160f4d-e50d-4aba-b894-29753ab99b81`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:17,664::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:17,666::task::595::Storage.TaskManager.Task::(_updateState) Task=`29d0c535-b7ae-4b49-ae90-4d26dac97b4b`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:17,666::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:17,667::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:17,668::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:17,668::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:17,669::task::1191::Storage.TaskManager.Task::(prepare) Task=`29d0c535-b7ae-4b49-ae90-4d26dac97b4b`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:17,669::task::595::Storage.TaskManager.Task::(_updateState) Task=`29d0c535-b7ae-4b49-ae90-4d26dac97b4b`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:17,670::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:17,671::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:17,671::task::993::Storage.TaskManager.Task::(_decref) Task=`29d0c535-b7ae-4b49-ae90-4d26dac97b4b`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:18,677::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:18,677::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dae0170-1eb6-4bc2-ad93-18f690375a30`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:18,678::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:18,678::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:18,678::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:18,678::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:18,678::task::1191::Storage.TaskManager.Task::(prepare) Task=`2dae0170-1eb6-4bc2-ad93-18f690375a30`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:18,678::task::595::Storage.TaskManager.Task::(_updateState) Task=`2dae0170-1eb6-4bc2-ad93-18f690375a30`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:18,678::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:18,678::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:18,678::task::993::Storage.TaskManager.Task::(_decref) Task=`2dae0170-1eb6-4bc2-ad93-18f690375a30`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:19,682::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:19,682::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e00441c-ccdb-48fe-855f-8af4153629de`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:19,683::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:19,683::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:19,683::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:19,683::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:19,683::task::1191::Storage.TaskManager.Task::(prepare) Task=`0e00441c-ccdb-48fe-855f-8af4153629de`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:19,683::task::595::Storage.TaskManager.Task::(_updateState) Task=`0e00441c-ccdb-48fe-855f-8af4153629de`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:19,683::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:19,683::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:19,683::task::993::Storage.TaskManager.Task::(_decref) Task=`0e00441c-ccdb-48fe-855f-8af4153629de`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:20,688::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:20,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfa92f92-f51c-4209-ba23-545bdbddb371`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:20,688::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:20,688::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:20,688::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:20,688::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:20,688::task::1191::Storage.TaskManager.Task::(prepare) Task=`bfa92f92-f51c-4209-ba23-545bdbddb371`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:20,688::task::595::Storage.TaskManager.Task::(_updateState) Task=`bfa92f92-f51c-4209-ba23-545bdbddb371`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:20,688::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:20,688::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:20,688::task::993::Storage.TaskManager.Task::(_decref) Task=`bfa92f92-f51c-4209-ba23-545bdbddb371`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:21,692::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:21,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`6eabc96b-2c62-4440-ac29-23b016e14d27`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:21,693::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:21,693::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:21,693::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:21,693::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:21,693::task::1191::Storage.TaskManager.Task::(prepare) Task=`6eabc96b-2c62-4440-ac29-23b016e14d27`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:21,693::task::595::Storage.TaskManager.Task::(_updateState) Task=`6eabc96b-2c62-4440-ac29-23b016e14d27`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:21,694::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:21,694::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:21,694::task::993::Storage.TaskManager.Task::(_decref) Task=`6eabc96b-2c62-4440-ac29-23b016e14d27`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:22,697::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:22,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`0795f6fa-a55a-482e-ad3b-447368519b15`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:22,697::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:22,698::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:22,698::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:22,698::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:22,698::task::1191::Storage.TaskManager.Task::(prepare) Task=`0795f6fa-a55a-482e-ad3b-447368519b15`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:22,698::task::595::Storage.TaskManager.Task::(_updateState) Task=`0795f6fa-a55a-482e-ad3b-447368519b15`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:22,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:22,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:22,698::task::993::Storage.TaskManager.Task::(_decref) Task=`0795f6fa-a55a-482e-ad3b-447368519b15`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:23,702::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:23,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`12cb806d-1677-4e44-bd11-e6d443d51e97`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:23,702::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:23,703::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:23,703::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:23,703::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:23,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`12cb806d-1677-4e44-bd11-e6d443d51e97`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:23,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`12cb806d-1677-4e44-bd11-e6d443d51e97`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:23,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:23,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:23,703::task::993::Storage.TaskManager.Task::(_decref) Task=`12cb806d-1677-4e44-bd11-e6d443d51e97`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:24,707::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:24,707::task::595::Storage.TaskManager.Task::(_updateState) Task=`737af434-5720-4a5a-9e21-c161e2bc687e`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:24,707::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:24,708::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:24,708::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:24,708::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:24,708::task::1191::Storage.TaskManager.Task::(prepare) Task=`737af434-5720-4a5a-9e21-c161e2bc687e`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:24,708::task::595::Storage.TaskManager.Task::(_updateState) Task=`737af434-5720-4a5a-9e21-c161e2bc687e`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:24,708::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:24,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:24,708::task::993::Storage.TaskManager.Task::(_decref) Task=`737af434-5720-4a5a-9e21-c161e2bc687e`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:25,712::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:25,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`15319de6-a167-49c2-abe1-67ff33cde07d`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:25,712::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:25,712::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:25,712::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:25,713::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:25,713::task::1191::Storage.TaskManager.Task::(prepare) Task=`15319de6-a167-49c2-abe1-67ff33cde07d`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:25,713::task::595::Storage.TaskManager.Task::(_updateState) Task=`15319de6-a167-49c2-abe1-67ff33cde07d`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:25,713::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:25,713::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:25,713::task::993::Storage.TaskManager.Task::(_decref) Task=`15319de6-a167-49c2-abe1-67ff33cde07d`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:26,717::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:26,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e8cfbf4-7b58-4026-91cb-b56a3007b697`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:26,717::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:26,717::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:26,717::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:26,717::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:26,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e8cfbf4-7b58-4026-91cb-b56a3007b697`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:26,718::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e8cfbf4-7b58-4026-91cb-b56a3007b697`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:26,718::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:26,718::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:26,718::task::993::Storage.TaskManager.Task::(_decref) Task=`5e8cfbf4-7b58-4026-91cb-b56a3007b697`::ref 0 aborting False >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,621::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb successfully acquired (id: 1) >34d66b4e-9038-4627-ac50-4802a327481c::INFO::2014-10-23 16:52:27,625::clusterlock::295::Storage.SANLock::(acquire) Acquiring cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb (id: 1) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::clusterlock::321::Storage.SANLock::(acquire) Cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb successfully acquired (id: 1) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::sp::272::Storage.StoragePool::(startSpm) spm lock acquired successfully >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::spbackends::422::Storage.StoragePoolMemoryBackend::(setSpmStatus) this storage pool implementation ignores the set spm status requests (lver=0, spmid=1) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7f2c7f2b-1e64-482a-a1cd-245d31cbfedc`::Request was made in '/usr/share/vdsm/storage/sp.py' line '426' at '_upgradePool' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,658::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`7f2c7f2b-1e64-482a-a1cd-245d31cbfedc`::Granted request >34d66b4e-9038-4627-ac50-4802a327481c::INFO::2014-10-23 16:52:27,659::sp::432::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `8851d365-d6eb-4bd4-b817-e27df4b7becb` >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,659::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`7ae653a5-06e1-4922-8d8d-1e60786ae88e`::Request was made in '/usr/share/vdsm/storage/sp.py' line '434' at '_upgradePool' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,659::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' for lock type 'exclusive' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,659::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free. Now locking as 'exclusive' (1 active user) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,659::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`7ae653a5-06e1-4922-8d8d-1e60786ae88e`::Granted request >Thread-41::DEBUG::2014-10-23 16:52:27,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,660::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,661::__init__::232::IOProcess::(_processLogs) (50738) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:27,661::__init__::232::IOProcess::(_processLogs) (50738) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,662::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:52:27,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,662::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,663::__init__::232::IOProcess::(_processLogs) (50739) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:52:27,663::__init__::232::IOProcess::(_processLogs) (50739) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,664::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:52:27,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,664::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,664::__init__::232::IOProcess::(_processLogs) (50740) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:52:27,664::__init__::232::IOProcess::(_processLogs) (50740) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,665::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-41::DEBUG::2014-10-23 16:52:27,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,665::__init__::232::IOProcess::(_processLogs) (50741) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:27,665::__init__::232::IOProcess::(_processLogs) (50741) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,666::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,666::__init__::232::IOProcess::(_processLogs) (50742) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:27,666::__init__::232::IOProcess::(_processLogs) (50742) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,666::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >34d66b4e-9038-4627-ac50-4802a327481c::INFO::2014-10-23 16:52:27,667::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >34d66b4e-9038-4627-ac50-4802a327481c::INFO::2014-10-23 16:52:27,667::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' (0 active users) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free, finding out if anyone is waiting for it. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', Clearing records. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::sp::437::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::sp::444::Storage.StoragePool::(_upgradePool) Registering with state change event >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::sp::447::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,667::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,668::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,668::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,668::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-41::DEBUG::2014-10-23 16:52:27,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,668::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,668::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,668::__init__::232::IOProcess::(_processLogs) (50743) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:27,668::__init__::232::IOProcess::(_processLogs) (50743) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,669::sd::647::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/master/vms >Thread-41::DEBUG::2014-10-23 16:52:27,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,669::__init__::232::IOProcess::(_processLogs) (50744) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,669::__init__::232::IOProcess::(_processLogs) (50744) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,670::__init__::232::IOProcess::(_processLogs) (50745) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,670::__init__::232::IOProcess::(_processLogs) (50745) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) (50746) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) (50746) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) (50747) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) (50747) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) (50748) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,672::__init__::232::IOProcess::(_processLogs) (50748) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,673::__init__::232::IOProcess::(_processLogs) (50749) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,673::__init__::232::IOProcess::(_processLogs) (50749) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) (50750) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) (50750) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) (50751) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) (50751) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,675::sd::650::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb/master/tasks >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) (50752) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) (50752) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) (50753) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,676::__init__::232::IOProcess::(_processLogs) (50753) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,677::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,677::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,677::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,677::__init__::232::IOProcess::(_processLogs) (50754) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,677::__init__::232::IOProcess::(_processLogs) (50754) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) (50755) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) (50755) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,678::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) (50756) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) (50756) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) (50757) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,679::__init__::232::IOProcess::(_processLogs) (50757) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,680::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,680::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,680::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,680::__init__::232::IOProcess::(_processLogs) (50758) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,680::__init__::232::IOProcess::(_processLogs) (50758) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,681::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,681::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,681::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,681::__init__::232::IOProcess::(_processLogs) (50759) Got request for method 'mkdir' >Thread-41::DEBUG::2014-10-23 16:52:27,681::__init__::232::IOProcess::(_processLogs) (50759) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,682::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,682::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,682::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,682::__init__::232::IOProcess::(_processLogs) (50760) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:27,683::__init__::232::IOProcess::(_processLogs) (50760) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:27,683::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:27,683::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:27,683::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:27,683::__init__::232::IOProcess::(_processLogs) (50761) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:27,684::__init__::232::IOProcess::(_processLogs) (50761) Queuing response >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,684::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,685::storage_mailbox::574::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1'] >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,686::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) dd of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000 (cwd None) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,705::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0131833 s, 77.7 MB/s\n'; <rc> = 0 >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,705::storage_mailbox::582::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,706::sp::319::Storage.StoragePool::(startSpm) SPM mailbox ready for pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe on master domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Dummy-49300::DEBUG::2014-10-23 16:52:27,706::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,706::sp::327::Storage.StoragePool::(startSpm) ended. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,712::task::595::Storage.TaskManager.Task::(_updateState) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::moving from state running -> state finished >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,712::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,712::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::task::916::Storage.TaskManager.Task::(_runJobs) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::Task.run: exit - success: result >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,713::task::993::Storage.TaskManager.Task::(_decref) Task=`34d66b4e-9038-4627-ac50-4802a327481c`::ref 0 aborting False >34d66b4e-9038-4627-ac50-4802a327481c::DEBUG::2014-10-23 16:52:27,714::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 0 >Thread-49192::DEBUG::2014-10-23 16:52:27,723::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:27,724::task::595::Storage.TaskManager.Task::(_updateState) Task=`9aa03385-b2e3-477c-88b9-f74d7392b96a`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:27,725::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:27,725::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:27,726::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'} >Thread-49192::INFO::2014-10-23 16:52:27,726::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:27,726::task::1191::Storage.TaskManager.Task::(prepare) Task=`9aa03385-b2e3-477c-88b9-f74d7392b96a`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '34d66b4e-9038-4627-ac50-4802a327481c'}} >Thread-49192::DEBUG::2014-10-23 16:52:27,727::task::595::Storage.TaskManager.Task::(_updateState) Task=`9aa03385-b2e3-477c-88b9-f74d7392b96a`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:27,727::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:27,728::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:27,728::task::993::Storage.TaskManager.Task::(_decref) Task=`9aa03385-b2e3-477c-88b9-f74d7392b96a`::ref 0 aborting False >Dummy-49300::DEBUG::2014-10-23 16:52:27,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0186632 s, 54.9 MB/s\n'; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:27,756::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:27,757::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b816204-b6f1-48d5-bf96-4408ec3ce541`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:27,758::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', options=None) >Thread-49192::INFO::2014-10-23 16:52:27,773::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}} >Thread-49192::DEBUG::2014-10-23 16:52:27,773::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b816204-b6f1-48d5-bf96-4408ec3ce541`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 4L}} >Thread-49192::DEBUG::2014-10-23 16:52:27,773::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b816204-b6f1-48d5-bf96-4408ec3ce541`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:27,773::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:27,773::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:27,773::task::993::Storage.TaskManager.Task::(_decref) Task=`2b816204-b6f1-48d5-bf96-4408ec3ce541`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:27,819::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:27,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f3ac5af-341d-456c-a660-002529cdb40f`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:27,819::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID='34d66b4e-9038-4627-ac50-4802a327481c', spUUID=None, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:27,819::taskManager::171::Storage.TaskManager::(clearTask) Entry. taskID: 34d66b4e-9038-4627-ac50-4802a327481c >Thread-49192::DEBUG::2014-10-23 16:52:27,820::taskManager::176::Storage.TaskManager::(clearTask) Return. >Thread-49192::INFO::2014-10-23 16:52:27,820::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-49192::DEBUG::2014-10-23 16:52:27,820::task::1191::Storage.TaskManager.Task::(prepare) Task=`6f3ac5af-341d-456c-a660-002529cdb40f`::finished: None >Thread-49192::DEBUG::2014-10-23 16:52:27,820::task::595::Storage.TaskManager.Task::(_updateState) Task=`6f3ac5af-341d-456c-a660-002529cdb40f`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:27,820::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:27,820::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:27,820::task::993::Storage.TaskManager.Task::(_decref) Task=`6f3ac5af-341d-456c-a660-002529cdb40f`::ref 0 aborting False >Detector thread::DEBUG::2014-10-23 16:52:27,971::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.25:55029 >Detector thread::DEBUG::2014-10-23 16:52:27,978::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.25:55029 >Detector thread::DEBUG::2014-10-23 16:52:27,978::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol xml from 10.34.63.25:55029 >Detector thread::DEBUG::2014-10-23 16:52:27,978::BindingXMLRPC::1172::XmlDetector::(handleSocket) xml over http detected from ('10.34.63.25', 55029) >Thread-49301::DEBUG::2014-10-23 16:52:27,979::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49301::DEBUG::2014-10-23 16:52:27,979::task::595::Storage.TaskManager.Task::(_updateState) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::moving from state init -> state preparing >Thread-49301::INFO::2014-10-23 16:52:27,980::logUtils::44::dispatcher::(wrapper) Run and protect: destroyStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, options=None) >Thread-49301::INFO::2014-10-23 16:52:27,980::hsm::1153::Storage.HSM::(destroyStoragePool) spUUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49301::DEBUG::2014-10-23 16:52:27,980::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a6d6db2e-8d38-4ae9-8f70-175f65efc537`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1159' at 'destroyStoragePool' >Thread-49301::DEBUG::2014-10-23 16:52:27,980::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49301::DEBUG::2014-10-23 16:52:27,980::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49301::DEBUG::2014-10-23 16:52:27,980::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`a6d6db2e-8d38-4ae9-8f70-175f65efc537`::Granted request >Thread-49301::DEBUG::2014-10-23 16:52:27,981::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::_resourcesAcquired: Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe (exclusive) >Thread-49301::DEBUG::2014-10-23 16:52:27,981::task::993::Storage.TaskManager.Task::(_decref) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::ref 1 aborting False >Thread-49301::DEBUG::2014-10-23 16:52:27,982::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`0b063a56-024b-4b5a-91f2-3b582e52c76f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1163' at 'destroyStoragePool' >Thread-49301::DEBUG::2014-10-23 16:52:27,982::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' for lock type 'exclusive' >Thread-49301::DEBUG::2014-10-23 16:52:27,982::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free. Now locking as 'exclusive' (1 active user) >Thread-49301::DEBUG::2014-10-23 16:52:27,982::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`0b063a56-024b-4b5a-91f2-3b582e52c76f`::Granted request >Thread-49301::DEBUG::2014-10-23 16:52:27,982::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::_resourcesAcquired: Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb (exclusive) >Thread-49301::DEBUG::2014-10-23 16:52:27,982::task::993::Storage.TaskManager.Task::(_decref) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::ref 1 aborting False >Thread-49301::WARNING::2014-10-23 16:52:27,982::sp::927::Storage.StoragePool::(forcedDetachSD) Force detaching domain `8851d365-d6eb-4bd4-b817-e27df4b7becb` >Thread-49301::INFO::2014-10-23 16:52:27,982::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 2 and domains map {} >Thread-49301::DEBUG::2014-10-23 16:52:27,982::sp::1143::Storage.StoragePool::(_cleanupDomainLinks) Removing: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49301::DEBUG::2014-10-23 16:52:27,983::sp::337::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`1fc04490-47c5-4c08-a598-dd6f0ad7c4f7`::Request was made in '/usr/share/vdsm/storage/sp.py' line '339' at '_shutDownUpgrade' >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' for lock type 'exclusive' >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free. Now locking as 'exclusive' (1 active user) >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe`ReqID=`1fc04490-47c5-4c08-a598-dd6f0ad7c4f7`::Granted request >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49301::DEBUG::2014-10-23 16:52:27,983::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49301::INFO::2014-10-23 16:52:27,984::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49301::DEBUG::2014-10-23 16:52:27,985::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb successfully released >Thread-49301::DEBUG::2014-10-23 16:52:27,986::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-49301::DEBUG::2014-10-23 16:52:27,986::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49301::INFO::2014-10-23 16:52:27,986::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49284::DEBUG::2014-10-23 16:52:27,986::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-49284::INFO::2014-10-23 16:52:27,986::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb (id: 1) >Thread-49284::DEBUG::2014-10-23 16:52:29,986::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 8851d365-d6eb-4bd4-b817-e27df4b7becb released successfully (id: 1) >Thread-49301::DEBUG::2014-10-23 16:52:29,987::sp::1407::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `11cfd6be-5798-4973-9e8b-a3262a2d3ebe` stopped monitoring domain `8851d365-d6eb-4bd4-b817-e27df4b7becb` >Thread-49301::DEBUG::2014-10-23 16:52:29,987::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49301::DEBUG::2014-10-23 16:52:29,987::sp::944::Storage.StoragePool::(forcedDetachSD) Force detach for domain `8851d365-d6eb-4bd4-b817-e27df4b7becb` is done >Thread-49301::INFO::2014-10-23 16:52:29,987::sd::533::Storage.StorageDomain::(detach) detaching storage domain 8851d365-d6eb-4bd4-b817-e27df4b7becb from pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-41::DEBUG::2014-10-23 16:52:29,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:29,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:29,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:29,988::__init__::232::IOProcess::(_processLogs) (50762) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:52:29,988::__init__::232::IOProcess::(_processLogs) (50762) Queuing response >Thread-49301::DEBUG::2014-10-23 16:52:29,988::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=11cfd6be-5798-4973-9e8b-a3262a2d3ebe', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Master', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=9fa4a55944d674a30717b4d73a66222ef67ec3de'] >Thread-49301::DEBUG::2014-10-23 16:52:29,989::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-49301::DEBUG::2014-10-23 16:52:29,989::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49301::DEBUG::2014-10-23 16:52:29,989::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-49301::DEBUG::2014-10-23 16:52:29,989::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-49301::DEBUG::2014-10-23 16:52:29,989::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Regular', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=c4e37ac1d90fa6c65cccfb096b3a0a7107090b6b'] >Thread-41::DEBUG::2014-10-23 16:52:29,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:29,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:29,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:29,990::__init__::232::IOProcess::(_processLogs) (50763) Got request for method 'writefile' >Thread-41::DEBUG::2014-10-23 16:52:29,992::__init__::232::IOProcess::(_processLogs) (50763) Queuing response >Thread-49301::WARNING::2014-10-23 16:52:29,992::outOfProcess::221::Storage.oop::(rename) renaming a directory is not an atomic operation >Thread-41::DEBUG::2014-10-23 16:52:29,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:29,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:29,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:29,992::__init__::232::IOProcess::(_processLogs) (50764) Got request for method 'rename' >Thread-41::DEBUG::2014-10-23 16:52:29,994::__init__::232::IOProcess::(_processLogs) (50764) Queuing response >Thread-49301::DEBUG::2014-10-23 16:52:29,994::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-49301::DEBUG::2014-10-23 16:52:29,994::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`351c58b8-cdd5-4ccf-b80a-caba65cf58eb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1132' at '_disconnectPool' >Thread-49301::DEBUG::2014-10-23 16:52:29,994::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-49301::DEBUG::2014-10-23 16:52:29,994::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-49301::DEBUG::2014-10-23 16:52:29,994::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`351c58b8-cdd5-4ccf-b80a-caba65cf58eb`::Granted request >Thread-49301::INFO::2014-10-23 16:52:29,995::sp::657::Storage.StoragePool::(disconnect) Disconnect from the storage pool 11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Dummy-49300::INFO::2014-10-23 16:52:30,685::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-49192::DEBUG::2014-10-23 16:52:31,099::task::595::Storage.TaskManager.Task::(_updateState) Task=`5467da9c-54f1-432f-be6b-8e4c80f0ddff`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:31,099::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:52:31,100::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:52:31,100::task::1191::Storage.TaskManager.Task::(prepare) Task=`5467da9c-54f1-432f-be6b-8e4c80f0ddff`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:52:31,100::task::595::Storage.TaskManager.Task::(_updateState) Task=`5467da9c-54f1-432f-be6b-8e4c80f0ddff`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:31,100::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:31,100::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:31,100::task::993::Storage.TaskManager.Task::(_decref) Task=`5467da9c-54f1-432f-be6b-8e4c80f0ddff`::ref 0 aborting False >Thread-49293::INFO::2014-10-23 16:52:31,939::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-49293::INFO::2014-10-23 16:52:31,939::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-49293::DEBUG::2014-10-23 16:52:31,940::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-49293::DEBUG::2014-10-23 16:52:31,953::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.0061331 s, 668 kB/s\n'; <rc> = 0 >Thread-49301::DEBUG::2014-10-23 16:52:32,917::fileUtils::124::Storage.fileUtils::(cleanupdir) Removing directory: /rhev/data-center/11cfd6be-5798-4973-9e8b-a3262a2d3ebe >Thread-49301::DEBUG::2014-10-23 16:52:32,918::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-49301::DEBUG::2014-10-23 16:52:32,918::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-49301::DEBUG::2014-10-23 16:52:32,918::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-49301::DEBUG::2014-10-23 16:52:32,918::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-49301::INFO::2014-10-23 16:52:32,918::logUtils::47::dispatcher::(wrapper) Run and protect: destroyStoragePool, Return response: True >Thread-49301::DEBUG::2014-10-23 16:52:32,918::task::1191::Storage.TaskManager.Task::(prepare) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::finished: True >Thread-49301::DEBUG::2014-10-23 16:52:32,919::task::595::Storage.TaskManager.Task::(_updateState) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::moving from state preparing -> state finished >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb': < ResourceRef 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', isValid: 'True' obj: 'None'>, 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe': < ResourceRef 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', isValid: 'True' obj: 'None'>} >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' (0 active users) >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free, finding out if anyone is waiting for it. >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', Clearing records. >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' (0 active users) >Thread-49301::DEBUG::2014-10-23 16:52:32,919::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe' is free, finding out if anyone is waiting for it. >Thread-49301::DEBUG::2014-10-23 16:52:32,920::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.11cfd6be-5798-4973-9e8b-a3262a2d3ebe', Clearing records. >Thread-49301::DEBUG::2014-10-23 16:52:32,920::task::993::Storage.TaskManager.Task::(_decref) Task=`b86cd53b-d84a-4258-be72-6381f7e57eb3`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:33,037::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [25d966f2] >Thread-49192::DEBUG::2014-10-23 16:52:33,037::task::595::Storage.TaskManager.Task::(_updateState) Task=`577991dc-a962-40f2-bf51-989e49274d7b`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:33,037::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID='11cfd6be-5798-4973-9e8b-a3262a2d3ebe', hostID=1, remove=False, options=None) >Thread-49192::WARNING::2014-10-23 16:52:33,037::hsm::1119::Storage.HSM::(disconnectStoragePool) disconnect sp: 11cfd6be-5798-4973-9e8b-a3262a2d3ebe failed. Known pools {} >Thread-49192::INFO::2014-10-23 16:52:33,037::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: None >Thread-49192::DEBUG::2014-10-23 16:52:33,038::task::1191::Storage.TaskManager.Task::(prepare) Task=`577991dc-a962-40f2-bf51-989e49274d7b`::finished: None >Thread-49192::DEBUG::2014-10-23 16:52:33,038::task::595::Storage.TaskManager.Task::(_updateState) Task=`577991dc-a962-40f2-bf51-989e49274d7b`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:33,038::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:33,038::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:33,038::task::993::Storage.TaskManager.Task::(_decref) Task=`577991dc-a962-40f2-bf51-989e49274d7b`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:33,068::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [7a9799dd] >Thread-49192::DEBUG::2014-10-23 16:52:33,068::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f1634c-1b4c-45fb-8500-8dd2dbab4f31`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:33,068::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:52:33,068::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:33,100::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:52:33,100::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:33,100::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:52:33,100::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:33,100::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:52:33,100::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:33,110::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:33,111::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:33,200::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:33,200::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,200::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,200::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,200::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,200::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,201::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:33,201::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:52:33,201::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:33,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`52f1634c-1b4c-45fb-8500-8dd2dbab4f31`::finished: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:33,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`52f1634c-1b4c-45fb-8500-8dd2dbab4f31`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:33,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:33,201::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:33,201::task::993::Storage.TaskManager.Task::(_decref) Task=`52f1634c-1b4c-45fb-8500-8dd2dbab4f31`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:39,696::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [e8224e1] >Thread-49192::DEBUG::2014-10-23 16:52:39,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`26073c09-e3f6-46a8-90f8-d4e2444b194c`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:39,702::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:52:39,704::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/10.34.63.199:_cmg03 >Thread-49192::DEBUG::2014-10-23 16:52:39,704::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t nfs -o soft,nosharecache,timeo=600,retrans=6,nfsvers=3 10.34.63.199:/cmg03 /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-41::DEBUG::2014-10-23 16:52:39,726::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,727::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,727::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,727::__init__::232::IOProcess::(_processLogs) (50765) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:39,727::__init__::232::IOProcess::(_processLogs) (50765) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,735::hsm::2372::Storage.HSM::(__prefetchDomains) nfs local path: /rhev/data-center/mnt/10.34.63.199:_cmg03 >Thread-41::DEBUG::2014-10-23 16:52:39,735::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,735::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,735::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,736::__init__::232::IOProcess::(_processLogs) (50766) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:39,736::__init__::232::IOProcess::(_processLogs) (50766) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,736::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: (u'8851d365-d6eb-4bd4-b817-e27df4b7becb',) >Thread-49192::DEBUG::2014-10-23 16:52:39,736::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {c53abbb1-bcd7-46ba-8717-53389331295d: storage.nfsSD.findDomain, 2027a9ce-c630-4748-bb7c-1e8cc9c88ca2: storage.blockSD.findDomain, 144ea0e9-c872-47d4-b323-bbc5685362a7: storage.nfsSD.findDomain, e9a9ddb7-0796-4ef3-8cb2-58e1bec3e3f9: storage.nfsSD.findDomain, 8851d365-d6eb-4bd4-b817-e27df4b7becb: storage.nfsSD.findDomain, 748c2d80-0413-4285-ae64-fade24166b4b: storage.nfsSD.findDomain, 2d4b6e8f-878f-4d9e-96f5-176b679b43e4: storage.blockSD.findDomain} >Thread-49192::INFO::2014-10-23 16:52:39,736::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:39,737::task::1191::Storage.TaskManager.Task::(prepare) Task=`26073c09-e3f6-46a8-90f8-d4e2444b194c`::finished: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:39,737::task::595::Storage.TaskManager.Task::(_updateState) Task=`26073c09-e3f6-46a8-90f8-d4e2444b194c`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:39,737::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:39,737::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:39,737::task::993::Storage.TaskManager.Task::(_decref) Task=`26073c09-e3f6-46a8-90f8-d4e2444b194c`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:39,753::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [e8224e1] >Thread-49192::DEBUG::2014-10-23 16:52:39,753::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:39,753::logUtils::44::dispatcher::(wrapper) Run and protect: formatStorageDomain(sdUUID='8851d365-d6eb-4bd4-b817-e27df4b7becb', autoDetach=False, options=None) >Thread-49192::DEBUG::2014-10-23 16:52:39,753::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:52:39,753::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:39,753::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:52:39,754::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:39,763::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:39,764::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:39,838::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:39,839::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`2c588800-fa4b-4a38-bd4b-45471705f745`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2747' at 'formatStorageDomain' >Thread-49192::DEBUG::2014-10-23 16:52:39,839::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' for lock type 'exclusive' >Thread-49192::DEBUG::2014-10-23 16:52:39,839::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free. Now locking as 'exclusive' (1 active user) >Thread-49192::DEBUG::2014-10-23 16:52:39,840::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb`ReqID=`2c588800-fa4b-4a38-bd4b-45471705f745`::Granted request >Thread-49192::DEBUG::2014-10-23 16:52:39,840::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::_resourcesAcquired: Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb (exclusive) >Thread-49192::DEBUG::2014-10-23 16:52:39,840::task::993::Storage.TaskManager.Task::(_decref) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::ref 1 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:39,840::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:52:39,840::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:39,840::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:52:39,840::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:39,840::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:52:39,840::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:39,855::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:39,856::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:39,945::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:39,945::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,945::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,945::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,945::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,945::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,946::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:39,946::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-41::DEBUG::2014-10-23 16:52:39,947::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,947::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,947::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,947::__init__::232::IOProcess::(_processLogs) (50767) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:39,947::__init__::232::IOProcess::(_processLogs) (50767) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,949::fileSD::152::Storage.StorageDomain::(__init__) Reading domain in path /rhev/data-center/mnt/10.34.63.199:_cmg03/8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:52:39,949::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,949::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,949::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,949::__init__::232::IOProcess::(_processLogs) (50768) Got request for method 'touch' >Thread-41::DEBUG::2014-10-23 16:52:39,950::__init__::232::IOProcess::(_processLogs) (50768) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,950::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with FileMetadataRW backend >Thread-41::DEBUG::2014-10-23 16:52:39,951::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,951::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,951::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,951::__init__::232::IOProcess::(_processLogs) (50769) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:52:39,952::__init__::232::IOProcess::(_processLogs) (50769) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,952::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Regular', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=c4e37ac1d90fa6c65cccfb096b3a0a7107090b6b'] >Thread-41::DEBUG::2014-10-23 16:52:39,953::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,953::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,953::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,953::__init__::232::IOProcess::(_processLogs) (50770) Got request for method 'access' >Thread-41::DEBUG::2014-10-23 16:52:39,953::__init__::232::IOProcess::(_processLogs) (50770) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,954::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,954::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,954::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,954::__init__::232::IOProcess::(_processLogs) (50771) Got request for method 'glob' >Thread-49192::DEBUG::2014-10-23 16:52:39,955::fileSD::610::Storage.StorageDomain::(imageGarbageCollector) Removing remnants of deleted images [] >Thread-41::DEBUG::2014-10-23 16:52:39,955::__init__::232::IOProcess::(_processLogs) (50771) Queuing response >Thread-49192::INFO::2014-10-23 16:52:39,955::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_imageNS already registered >Thread-49192::INFO::2014-10-23 16:52:39,956::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 8851d365-d6eb-4bd4-b817-e27df4b7becb_volumeNS already registered >Thread-41::DEBUG::2014-10-23 16:52:39,956::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,956::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,956::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,956::__init__::232::IOProcess::(_processLogs) (50772) Got request for method 'readfile' >Thread-41::DEBUG::2014-10-23 16:52:39,956::__init__::232::IOProcess::(_processLogs) (50772) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:39,957::persistentDict::234::Storage.PersistentDict::(refresh) read lines (FileMetadataRW)=['CLASS=Data', 'DESCRIPTION=nfs_0', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'POOL_UUID=', 'REMOTE_PATH=10.34.63.199:/cmg03', 'ROLE=Regular', 'SDUUID=8851d365-d6eb-4bd4-b817-e27df4b7becb', 'TYPE=NFS', 'VERSION=3', '_SHA_CKSUM=c4e37ac1d90fa6c65cccfb096b3a0a7107090b6b'] >Thread-49192::INFO::2014-10-23 16:52:39,957::fileSD::510::Storage.StorageDomain::(format) Formatting domain 8851d365-d6eb-4bd4-b817-e27df4b7becb >Thread-41::DEBUG::2014-10-23 16:52:39,957::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,958::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,958::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,958::__init__::232::IOProcess::(_processLogs) (50773) Got request for method 'glob' >Thread-41::DEBUG::2014-10-23 16:52:39,958::__init__::232::IOProcess::(_processLogs) (50773) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,959::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,959::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) (50774) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) (50774) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,960::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,961::__init__::232::IOProcess::(_processLogs) (50775) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,961::__init__::232::IOProcess::(_processLogs) (50775) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,961::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,961::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,961::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,962::__init__::232::IOProcess::(_processLogs) (50776) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:39,962::__init__::232::IOProcess::(_processLogs) (50776) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,962::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,962::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,962::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,963::__init__::232::IOProcess::(_processLogs) (50777) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,963::__init__::232::IOProcess::(_processLogs) (50777) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,963::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,963::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,963::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,964::__init__::232::IOProcess::(_processLogs) (50778) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,964::__init__::232::IOProcess::(_processLogs) (50778) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,965::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,965::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,965::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,965::__init__::232::IOProcess::(_processLogs) (50779) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,965::__init__::232::IOProcess::(_processLogs) (50779) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,966::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,966::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,966::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,966::__init__::232::IOProcess::(_processLogs) (50780) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,966::__init__::232::IOProcess::(_processLogs) (50780) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,967::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,967::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,967::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,967::__init__::232::IOProcess::(_processLogs) (50781) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,967::__init__::232::IOProcess::(_processLogs) (50781) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,968::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,968::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,968::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,968::__init__::232::IOProcess::(_processLogs) (50782) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,969::__init__::232::IOProcess::(_processLogs) (50782) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,969::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,969::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,969::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,969::__init__::232::IOProcess::(_processLogs) (50783) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,970::__init__::232::IOProcess::(_processLogs) (50783) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,970::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,970::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,970::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,970::__init__::232::IOProcess::(_processLogs) (50784) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,971::__init__::232::IOProcess::(_processLogs) (50784) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,972::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,972::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,972::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,972::__init__::232::IOProcess::(_processLogs) (50785) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,972::__init__::232::IOProcess::(_processLogs) (50785) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,973::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,973::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,973::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,973::__init__::232::IOProcess::(_processLogs) (50786) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,973::__init__::232::IOProcess::(_processLogs) (50786) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,974::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,974::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,974::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,974::__init__::232::IOProcess::(_processLogs) (50787) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:39,974::__init__::232::IOProcess::(_processLogs) (50787) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,975::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,975::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,975::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,975::__init__::232::IOProcess::(_processLogs) (50788) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,975::__init__::232::IOProcess::(_processLogs) (50788) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,976::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,976::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,976::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,976::__init__::232::IOProcess::(_processLogs) (50789) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:39,976::__init__::232::IOProcess::(_processLogs) (50789) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,977::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,977::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,977::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,977::__init__::232::IOProcess::(_processLogs) (50790) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,977::__init__::232::IOProcess::(_processLogs) (50790) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,978::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,978::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,978::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,978::__init__::232::IOProcess::(_processLogs) (50791) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:39,978::__init__::232::IOProcess::(_processLogs) (50791) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,979::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,979::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,979::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,979::__init__::232::IOProcess::(_processLogs) (50792) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,979::__init__::232::IOProcess::(_processLogs) (50792) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,980::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,980::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,980::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,980::__init__::232::IOProcess::(_processLogs) (50793) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) (50793) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) (50794) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,981::__init__::232::IOProcess::(_processLogs) (50794) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,982::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,982::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,982::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,982::__init__::232::IOProcess::(_processLogs) (50795) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,983::__init__::232::IOProcess::(_processLogs) (50795) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,983::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,984::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,984::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,984::__init__::232::IOProcess::(_processLogs) (50796) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,984::__init__::232::IOProcess::(_processLogs) (50796) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,985::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,985::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,985::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,985::__init__::232::IOProcess::(_processLogs) (50797) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,986::__init__::232::IOProcess::(_processLogs) (50797) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,986::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,986::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,986::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,986::__init__::232::IOProcess::(_processLogs) (50798) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:39,987::__init__::232::IOProcess::(_processLogs) (50798) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,987::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,987::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,987::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,988::__init__::232::IOProcess::(_processLogs) (50799) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,988::__init__::232::IOProcess::(_processLogs) (50799) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,989::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,989::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,989::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,989::__init__::232::IOProcess::(_processLogs) (50800) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) (50800) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) (50801) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,990::__init__::232::IOProcess::(_processLogs) (50801) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,991::__init__::232::IOProcess::(_processLogs) (50802) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,992::__init__::232::IOProcess::(_processLogs) (50802) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,992::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,992::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,993::__init__::232::IOProcess::(_processLogs) (50803) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,993::__init__::232::IOProcess::(_processLogs) (50803) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,993::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,994::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,994::__init__::232::IOProcess::(_processLogs) (50804) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) (50804) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) (50805) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,995::__init__::232::IOProcess::(_processLogs) (50805) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,996::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,996::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,996::__init__::232::IOProcess::(_processLogs) (50806) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:39,997::__init__::232::IOProcess::(_processLogs) (50806) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,998::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,998::__init__::232::IOProcess::(_processLogs) (50807) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:39,998::__init__::232::IOProcess::(_processLogs) (50807) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:39,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:39,999::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:39,999::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:39,999::__init__::232::IOProcess::(_processLogs) (50808) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:39,999::__init__::232::IOProcess::(_processLogs) (50808) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,000::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,000::__init__::232::IOProcess::(_processLogs) (50809) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) (50809) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) (50810) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,001::__init__::232::IOProcess::(_processLogs) (50810) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,002::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,002::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,002::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,002::__init__::232::IOProcess::(_processLogs) (50811) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,003::__init__::232::IOProcess::(_processLogs) (50811) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,004::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,004::__init__::232::IOProcess::(_processLogs) (50812) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,004::__init__::232::IOProcess::(_processLogs) (50812) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,005::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,005::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,005::__init__::232::IOProcess::(_processLogs) (50813) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,006::__init__::232::IOProcess::(_processLogs) (50813) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,006::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,006::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,006::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,007::__init__::232::IOProcess::(_processLogs) (50814) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,007::__init__::232::IOProcess::(_processLogs) (50814) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,008::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,008::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,008::__init__::232::IOProcess::(_processLogs) (50815) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) (50815) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) (50816) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,009::__init__::232::IOProcess::(_processLogs) (50816) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,010::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,010::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,010::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,010::__init__::232::IOProcess::(_processLogs) (50817) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,011::__init__::232::IOProcess::(_processLogs) (50817) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,012::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,012::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,012::__init__::232::IOProcess::(_processLogs) (50818) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,012::__init__::232::IOProcess::(_processLogs) (50818) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,013::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,013::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,013::__init__::232::IOProcess::(_processLogs) (50819) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,014::__init__::232::IOProcess::(_processLogs) (50819) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,014::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,014::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,015::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,015::__init__::232::IOProcess::(_processLogs) (50820) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,015::__init__::232::IOProcess::(_processLogs) (50820) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,016::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,016::__init__::232::IOProcess::(_processLogs) (50821) Got request for method 'unlink' >Thread-41::DEBUG::2014-10-23 16:52:40,017::__init__::232::IOProcess::(_processLogs) (50821) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,017::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,017::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,017::__init__::232::IOProcess::(_processLogs) (50822) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:40,018::__init__::232::IOProcess::(_processLogs) (50822) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,018::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,018::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,018::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,018::__init__::232::IOProcess::(_processLogs) (50823) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:40,019::__init__::232::IOProcess::(_processLogs) (50823) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,019::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,019::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,019::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,020::__init__::232::IOProcess::(_processLogs) (50824) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,020::__init__::232::IOProcess::(_processLogs) (50824) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,020::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,021::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,021::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,021::__init__::232::IOProcess::(_processLogs) (50825) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:40,021::__init__::232::IOProcess::(_processLogs) (50825) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,022::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,022::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,022::__init__::232::IOProcess::(_processLogs) (50826) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,022::__init__::232::IOProcess::(_processLogs) (50826) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,023::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,023::__init__::232::IOProcess::(_processLogs) (50827) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:40,024::__init__::232::IOProcess::(_processLogs) (50827) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,024::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,024::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,024::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,024::__init__::232::IOProcess::(_processLogs) (50828) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:40,025::__init__::232::IOProcess::(_processLogs) (50828) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,025::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,025::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,026::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,026::__init__::232::IOProcess::(_processLogs) (50829) Got request for method 'stat' >Thread-41::DEBUG::2014-10-23 16:52:40,026::__init__::232::IOProcess::(_processLogs) (50829) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,027::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,027::__init__::232::IOProcess::(_processLogs) (50830) Got request for method 'listdir' >Thread-41::DEBUG::2014-10-23 16:52:40,027::__init__::232::IOProcess::(_processLogs) (50830) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,028::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,028::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,028::__init__::232::IOProcess::(_processLogs) (50831) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:40,029::__init__::232::IOProcess::(_processLogs) (50831) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,029::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,029::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,029::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,029::__init__::232::IOProcess::(_processLogs) (50832) Got request for method 'rmdir' >Thread-41::DEBUG::2014-10-23 16:52:40,030::__init__::232::IOProcess::(_processLogs) (50832) Queuing response >Thread-41::DEBUG::2014-10-23 16:52:40,030::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-41::DEBUG::2014-10-23 16:52:40,030::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-41::DEBUG::2014-10-23 16:52:40,030::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-41::DEBUG::2014-10-23 16:52:40,030::__init__::232::IOProcess::(_processLogs) (50833) Got request for method 'rmdir' >Thread-49192::INFO::2014-10-23 16:52:40,031::logUtils::47::dispatcher::(wrapper) Run and protect: formatStorageDomain, Return response: None >Thread-49192::DEBUG::2014-10-23 16:52:40,031::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::finished: None >Thread-49192::DEBUG::2014-10-23 16:52:40,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb': < ResourceRef 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', isValid: 'True' obj: 'None'>} >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' (0 active users) >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb' is free, finding out if anyone is waiting for it. >Thread-49192::DEBUG::2014-10-23 16:52:40,032::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.8851d365-d6eb-4bd4-b817-e27df4b7becb', Clearing records. >Thread-49192::DEBUG::2014-10-23 16:52:40,032::task::993::Storage.TaskManager.Task::(_decref) Task=`2b97e4ac-f187-4440-9b20-1ab5156943d1`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 16:52:40,031::__init__::232::IOProcess::(_processLogs) (50833) Queuing response >Thread-49192::DEBUG::2014-10-23 16:52:40,084::BindingXMLRPC::318::vds::(wrapper) client [10.34.63.25] flowID [373890e3] >Thread-49192::DEBUG::2014-10-23 16:52:40,084::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ed6ed41-cf63-4277-9582-3157af726b14`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:40,084::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=1, spUUID='00000000-0000-0000-0000-000000000000', conList=[{'connection': '10.34.63.199:/cmg03', 'iqn': '', 'user': '', 'tpgt': '1', 'password': '******', 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0', 'port': ''}], options=None) >Thread-49192::DEBUG::2014-10-23 16:52:40,085::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount -f -l /rhev/data-center/mnt/10.34.63.199:_cmg03 (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:40,109::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-49192::DEBUG::2014-10-23 16:52:40,109::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:40,109::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-49192::DEBUG::2014-10-23 16:52:40,109::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-49192::DEBUG::2014-10-23 16:52:40,109::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-49192::DEBUG::2014-10-23 16:52:40,110::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:40,119::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::DEBUG::2014-10-23 16:52:40,121::multipath::110::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-49192::DEBUG::2014-10-23 16:52:40,211::multipath::110::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49192::DEBUG::2014-10-23 16:52:40,212::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-49192::INFO::2014-10-23 16:52:40,213::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:40,213::task::1191::Storage.TaskManager.Task::(prepare) Task=`7ed6ed41-cf63-4277-9582-3157af726b14`::finished: {'statuslist': [{'status': 0, 'id': '28d7ce9c-511a-4e66-ab62-cc3452ef3ec0'}]} >Thread-49192::DEBUG::2014-10-23 16:52:40,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`7ed6ed41-cf63-4277-9582-3157af726b14`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:40,213::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:40,213::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:40,213::task::993::Storage.TaskManager.Task::(_decref) Task=`7ed6ed41-cf63-4277-9582-3157af726b14`::ref 0 aborting False >Thread-49192::DEBUG::2014-10-23 16:52:46,438::task::595::Storage.TaskManager.Task::(_updateState) Task=`998ee7be-6fb9-4481-bcfc-fe58400558cb`::moving from state init -> state preparing >Thread-49192::INFO::2014-10-23 16:52:46,438::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-49192::INFO::2014-10-23 16:52:46,438::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-49192::DEBUG::2014-10-23 16:52:46,439::task::1191::Storage.TaskManager.Task::(prepare) Task=`998ee7be-6fb9-4481-bcfc-fe58400558cb`::finished: {} >Thread-49192::DEBUG::2014-10-23 16:52:46,439::task::595::Storage.TaskManager.Task::(_updateState) Task=`998ee7be-6fb9-4481-bcfc-fe58400558cb`::moving from state preparing -> state finished >Thread-49192::DEBUG::2014-10-23 16:52:46,439::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49192::DEBUG::2014-10-23 16:52:46,439::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49192::DEBUG::2014-10-23 16:52:46,439::task::993::Storage.TaskManager.Task::(_decref) Task=`998ee7be-6fb9-4481-bcfc-fe58400558cb`::ref 0 aborting False >MainThread::DEBUG::2014-10-23 17:01:09,458::vdsm::58::vds::(sigtermHandler) Received signal 15 >MainThread::DEBUG::2014-10-23 17:01:09,464::protocoldetector::135::vds.MultiProtocolAcceptor::(stop) Stopping Acceptor >MainThread::INFO::2014-10-23 17:01:09,464::__init__::563::jsonrpc.JsonRpcServer::(stop) Stopping JsonRPC Server >Detector thread::DEBUG::2014-10-23 17:01:09,464::protocoldetector::106::vds.MultiProtocolAcceptor::(_cleanup) Cleaning Acceptor >MainThread::INFO::2014-10-23 17:01:09,465::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-23 17:01:09,465::momIF::91::MOM::(stop) Shutting down MOM >MainThread::DEBUG::2014-10-23 17:01:09,466::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bf9f5a-53bd-4d39-8cdd-28d79f4dbdc9`::moving from state init -> state preparing >MainThread::INFO::2014-10-23 17:01:09,466::logUtils::44::dispatcher::(wrapper) Run and protect: prepareForShutdown(options=None) >Thread-12::DEBUG::2014-10-23 17:01:09,467::storageServer::706::Storage.ConnectionMonitor::(_monitorConnections) Monitoring stopped >MainThread::INFO::2014-10-23 17:01:09,485::domainMonitor::137::Storage.DomainMonitor::(close) Stopping domain monitors >MainThread::DEBUG::2014-10-23 17:01:09,485::taskManager::90::Storage.TaskManager::(prepareForShutdown) Request to stop all tasks >MainThread::INFO::2014-10-23 17:01:09,485::logUtils::47::dispatcher::(wrapper) Run and protect: prepareForShutdown, Return response: None >MainThread::DEBUG::2014-10-23 17:01:09,485::task::1191::Storage.TaskManager.Task::(prepare) Task=`c9bf9f5a-53bd-4d39-8cdd-28d79f4dbdc9`::finished: None >MainThread::DEBUG::2014-10-23 17:01:09,485::task::595::Storage.TaskManager.Task::(_updateState) Task=`c9bf9f5a-53bd-4d39-8cdd-28d79f4dbdc9`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-23 17:01:09,485::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-23 17:01:09,485::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-23 17:01:09,485::task::993::Storage.TaskManager.Task::(_decref) Task=`c9bf9f5a-53bd-4d39-8cdd-28d79f4dbdc9`::ref 0 aborting False >MainThread::INFO::2014-10-23 17:01:09,485::vdsm::137::vds::(run) VDSM main thread ended. Waiting for 19 other threads... >MainThread::INFO::2014-10-23 17:01:09,485::vdsm::141::vds::(run) <_MainThread(MainThread, started 139815517505280)> >MainThread::INFO::2014-10-23 17:01:09,485::vdsm::141::vds::(run) <Thread(Thread-1, started daemon 139815425320704)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <Thread(libvirtEventLoop, started daemon 139815345452800)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(34d66b4e-9038-4627-ac50-4802a327481c, started daemon 139815045547776)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <PolicyEngine(PolicyEngine, started daemon 139813770479360)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(Thread-6, started daemon 139815303493376)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(87751498-570b-40e4-98f5-e62cda3820c3, started daemon 139815293003520)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(04c513ce-0e7c-41da-869c-b7c70c632ec8, started daemon 139815035057920)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(f3bdf0c6-bcef-4cd3-ac35-c933e65bcf0e, started daemon 139815334962944)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(c0c4427f-0aca-4388-b130-2326c963f188, started daemon 139815313983232)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(Thread-9, started daemon 139815056037632)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(Thread-8, started daemon 139815066527488)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <WorkerThread(5f5f57b7-4e99-4671-bffb-93d5961d3afc, started daemon 139815077017344)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <HostMonitor(HostMonitor, started daemon 139814229645056)> >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <Thread(Thread-41, started daemon 139813759989504)> >MainThread::INFO::2014-10-23 17:01:09,486::momIF::91::MOM::(stop) Shutting down MOM >MainThread::INFO::2014-10-23 17:01:09,486::vdsm::141::vds::(run) <MomThread(MOM, started 139814271710976)> >MainThread::INFO::2014-10-23 17:01:09,487::vmchannels::188::vds::(stop) VM channels listener was stopped. >MainThread::INFO::2014-10-23 17:01:09,487::vdsm::141::vds::(run) <Listener(VM Channels Listener, started daemon 139814250731264)> >MainThread::INFO::2014-10-23 17:01:09,487::vdsm::141::vds::(run) <GuestManager(GuestManager, started daemon 139813791459072)> >MainThread::INFO::2014-10-23 17:01:09,487::vdsm::141::vds::(run) <_DummyThread(Dummy-49300, started daemon 139812877084416)> >MainThread::INFO::2014-10-23 17:01:09,487::vdsm::141::vds::(run) <WorkerThread(bce82ba2-140e-4563-8f2e-d6c8432a1d0d, started daemon 139815324473088)> >VM Channels Listener::INFO::2014-10-23 17:01:10,145::vmchannels::183::vds::(run) VM channels listener thread has ended. >MainThread::WARNING::2014-10-23 17:12:41,660::vdsmDebugPlugin::40::DebugInterpreter::(__turnOnDebugPlugin) Starting Debug Interpreter. Tread lightly! >MainThread::INFO::2014-10-23 17:12:41,661::vdsm::131::vds::(run) (PID: 9653) I am the actual vdsm 4.16.7.1-1.el6ev dell-r210ii-14.rhev.lab.eng.brq.redhat.com (2.6.32-431.el6.x86_64) >MainThread::DEBUG::2014-10-23 17:12:41,662::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'Storage' >MainThread::DEBUG::2014-10-23 17:12:41,662::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 10, waitTimeout: 3, maxTasks: 500 >MainThread::DEBUG::2014-10-23 17:12:41,663::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt >MainThread::WARNING::2014-10-23 17:12:41,664::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt already exists >MainThread::DEBUG::2014-10-23 17:12:41,725::supervdsm::77::SuperVdsmProxy::(_connect) Trying to connect to Super Vdsm >MainThread::DEBUG::2014-10-23 17:12:41,735::multipath::145::Storage.Multipath::(isEnabled) Current revision of multipath.conf detected, preserving >MainThread::DEBUG::2014-10-23 17:12:41,735::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) /usr/bin/sudo -n /sbin/lvm dumpconfig global/locking_type (cwd None) >MainThread::DEBUG::2014-10-23 17:12:41,841::hsm::418::Storage.Misc.excCmd::(__validateLvmLockingType) SUCCESS: <err> = ''; <rc> = 0 >MainThread::DEBUG::2014-10-23 17:12:41,842::hsm::442::Storage.HSM::(__cleanStorageRepository) Started cleaning storage repository at '/rhev/data-center' >MainThread::DEBUG::2014-10-23 17:12:41,844::hsm::474::Storage.HSM::(__cleanStorageRepository) White list: ['/rhev/data-center/hsm-tasks', '/rhev/data-center/hsm-tasks/*', '/rhev/data-center/mnt'] >MainThread::DEBUG::2014-10-23 17:12:41,844::hsm::475::Storage.HSM::(__cleanStorageRepository) Mount list: [] >MainThread::DEBUG::2014-10-23 17:12:41,844::hsm::477::Storage.HSM::(__cleanStorageRepository) Cleaning leftovers >MainThread::DEBUG::2014-10-23 17:12:41,845::hsm::520::Storage.HSM::(__cleanStorageRepository) Finished cleaning storage repository at '/rhev/data-center' >storageRefresh::DEBUG::2014-10-23 17:12:41,845::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >MainThread::INFO::2014-10-23 17:12:41,848::dispatcher::46::Storage.Dispatcher::(__init__) Starting StorageDispatcher... >storageRefresh::DEBUG::2014-10-23 17:12:41,848::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-23 17:12:41,890::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >storageRefresh::DEBUG::2014-10-23 17:12:41,890::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >storageRefresh::DEBUG::2014-10-23 17:12:41,890::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >storageRefresh::DEBUG::2014-10-23 17:12:41,891::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >MainThread::DEBUG::2014-10-23 17:12:41,894::task::595::Storage.TaskManager.Task::(_updateState) Task=`24aa725f-151a-43ae-93c5-4d72c6eead0a`::moving from state init -> state preparing >MainThread::INFO::2014-10-23 17:12:41,896::logUtils::44::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback(callbackFunc=<bound method clientIF.contEIOVms of <clientIF.clientIF object at 0x2fb8e90>>) >MainThread::INFO::2014-10-23 17:12:41,897::logUtils::47::dispatcher::(wrapper) Run and protect: registerDomainStateChangeCallback, Return response: None >MainThread::DEBUG::2014-10-23 17:12:41,897::task::1191::Storage.TaskManager.Task::(prepare) Task=`24aa725f-151a-43ae-93c5-4d72c6eead0a`::finished: None >MainThread::DEBUG::2014-10-23 17:12:41,897::task::595::Storage.TaskManager.Task::(_updateState) Task=`24aa725f-151a-43ae-93c5-4d72c6eead0a`::moving from state preparing -> state finished >MainThread::DEBUG::2014-10-23 17:12:41,897::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >MainThread::DEBUG::2014-10-23 17:12:41,897::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >MainThread::DEBUG::2014-10-23 17:12:41,897::task::993::Storage.TaskManager.Task::(_decref) Task=`24aa725f-151a-43ae-93c5-4d72c6eead0a`::ref 0 aborting False >MainThread::INFO::2014-10-23 17:12:41,898::momIF::49::MOM::(__init__) Starting up MOM >MainThread::INFO::2014-10-23 17:12:41,899::vmchannels::192::vds::(settimeout) Setting channels' timeout to 30 seconds. >VM Channels Listener::INFO::2014-10-23 17:12:41,901::vmchannels::175::vds::(run) Starting VM channels listener thread. >clientIFinit::DEBUG::2014-10-23 17:12:41,903::libvirtconnection::150::root::(get) trying to connect libvirt >storageRefresh::DEBUG::2014-10-23 17:12:41,903::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-23 17:12:41,903::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >MainThread::DEBUG::2014-10-23 17:12:41,917::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <rpc.BindingXMLRPC.XmlDetector instance at 0x30f9b48> >MainThread::DEBUG::2014-10-23 17:12:41,979::protocoldetector::131::vds.MultiProtocolAcceptor::(add_detector) adding detector: <yajsonrpc.stompReactor.StompDetector instance at 0x329f2d8> >JsonRpcServer::DEBUG::2014-10-23 17:12:41,981::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Detector thread::DEBUG::2014-10-23 17:12:41,983::protocoldetector::70::vds.MultiProtocolAcceptor::(serve_forever) Acceptor running >Detector thread::DEBUG::2014-10-23 17:12:41,992::protocoldetector::72::vds.MultiProtocolAcceptor::(serve_forever) Using required_size=7 >storageRefresh::DEBUG::2014-10-23 17:12:42,073::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-23 17:12:42,074::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,075::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,076::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,077::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,077::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,078::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,079::misc::751::Storage.SamplingMethod::(__call__) Returning last result >storageRefresh::DEBUG::2014-10-23 17:12:42,080::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,084::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >storageRefresh::DEBUG::2014-10-23 17:12:42,180::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >storageRefresh::DEBUG::2014-10-23 17:12:42,181::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,181::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,181::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-23 17:12:42,266::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-23 17:12:42,267::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,267::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags (cwd None) >storageRefresh::DEBUG::2014-10-23 17:12:42,344::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-23 17:12:42,344::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,344::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >storageRefresh::DEBUG::2014-10-23 17:12:42,423::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >storageRefresh::DEBUG::2014-10-23 17:12:42,423::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >storageRefresh::DEBUG::2014-10-23 17:12:42,423::hsm::387::Storage.HSM::(storageRefresh) HSM is ready >Detector thread::DEBUG::2014-10-23 17:12:42,638::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.25:44588 >Detector thread::DEBUG::2014-10-23 17:12:42,690::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.25:44588 >Detector thread::DEBUG::2014-10-23 17:12:42,690::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.34.63.25:44588 >Detector thread::DEBUG::2014-10-23 17:12:42,690::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-23 17:12:42,690::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-23 17:12:42,690::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.34.63.25', 44588) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:42,691::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:42,691::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:42,732::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:42,733::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-14::DEBUG::2014-10-23 17:12:42,733::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.ping' in bridge with {} >Thread-14::DEBUG::2014-10-23 17:12:42,733::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.ping' in bridge with True >Thread-14::DEBUG::2014-10-23 17:12:42,734::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:42,746::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:42,747::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-15::DEBUG::2014-10-23 17:12:42,747::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-15::DEBUG::2014-10-23 17:12:42,802::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,808::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,823::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,831::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,831::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,840::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,841::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,849::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,850::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,858::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,859::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,867::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,868::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,875::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,876::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,884::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,885::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/DK3h97rG8UzDohs/bonding/mode (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,892::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,893::utils::738::root::(execCmd) /usr/bin/sudo -n /usr/bin/tee /sys/class/net/bonding_masters (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,925::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,945::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-15::DEBUG::2014-10-23 17:12:42,947::utils::738::root::(execCmd) numactl --hardware (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,951::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,951::utils::738::root::(execCmd) sysctl -n -e kernel.numa_balancing (cwd None) >Thread-15::DEBUG::2014-10-23 17:12:42,955::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-15::DEBUG::2014-10-23 17:12:42,958::caps::269::root::(getLiveMergeSupport) libvirt is missing 'VIR_DOMAIN_BLOCK_COMMIT_RELATIVE': live merge disabled >Thread-15::DEBUG::2014-10-23 17:12:42,959::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:193293f409a'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.el6.x86_64', 'buildtime': 1384118394.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '1.el6ev', 'buildtime': 1413902785L, 'version': '4.16.7.1'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'mom': {'release': '2.el6', 'buildtime': 1403794344L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz', 'liveMerge': 'false', 'hooks': {'after_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_network_setup': {'30_ethtool_options': {'md5': '2f6fe7f77eb498fb9493ef9ea5ad8705'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '92498ed80c0219749829ecce2813fc7c'}}, 'before_vm_start': {'50_qemucmdline': {'md5': 'a884929ad6f5eb039887157288867409'}, '50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}, '10_faqemu': {'md5': 'e3cb338be6b9075a04bed0e7400ab7e2'}, '50_macspoof': {'md5': '25deea559772719b90117f2e1009d823'}}, 'after_device_migrate_destination': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'before_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_nic_hotunplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_device_destroy': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 0, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'gateway': '10.34.63.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'mtu': '1500', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ports': ['em1']}}, 'bridges': {'rhevm': {'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'ipv6gateway': 'fe80:52:0:223c::3fe', 'gateway': '10.34.63.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '20', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.d4ae52c71e3e', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.d4ae52c71e3e', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '20', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['em1']}}, 'uuid': '4C4C4544-0033-5310-8052-CAC04F4A354A', 'lastClientIface': 'rhevm', 'nics': {'em1': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': 'd4:ae:52:c7:1e:3e', 'DEVICE': 'em1', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3e', 'speed': 1000}, 'em2': {'addr': '', 'cfg': {'UUID': 'e32b7001-2514-471c-8c69-a075eed805e5', 'NM_CONTROLLED': 'yes', 'HWADDR': 'D4:AE:52:C7:1E:3F', 'BOOTPROTO': 'dhcp', 'DEVICE': 'em2', 'TYPE': 'Ethernet', 'ONBOOT': 'no'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3f', 'speed': 0}}, 'software_revision': '1', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:193293f409a', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '7842', 'cpuSpeed': '3301.000', 'numaNodes': {u'0': {'totalMemory': '7842', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-15::DEBUG::2014-10-23 17:12:42,962::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:43,440::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:43,441::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-16::DEBUG::2014-10-23 17:12:43,441::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-16::DEBUG::2014-10-23 17:12:43,443::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-16::DEBUG::2014-10-23 17:12:43,446::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-16::DEBUG::2014-10-23 17:12:43,467::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-16::DEBUG::2014-10-23 17:12:43,468::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:193293f409a'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.el6.x86_64', 'buildtime': 1384118394.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '1.el6ev', 'buildtime': 1413902785L, 'version': '4.16.7.1'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'mom': {'release': '2.el6', 'buildtime': 1403794344L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz', 'liveMerge': 'false', 'hooks': {'after_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_network_setup': {'30_ethtool_options': {'md5': '2f6fe7f77eb498fb9493ef9ea5ad8705'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '92498ed80c0219749829ecce2813fc7c'}}, 'before_vm_start': {'50_qemucmdline': {'md5': 'a884929ad6f5eb039887157288867409'}, '50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}, '10_faqemu': {'md5': 'e3cb338be6b9075a04bed0e7400ab7e2'}, '50_macspoof': {'md5': '25deea559772719b90117f2e1009d823'}}, 'after_device_migrate_destination': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'before_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_nic_hotunplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_device_destroy': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 0, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'gateway': '10.34.63.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'mtu': '1500', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ports': ['em1']}}, 'bridges': {'rhevm': {'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'ipv6gateway': 'fe80:52:0:223c::3fe', 'gateway': '10.34.63.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '167', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.d4ae52c71e3e', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.d4ae52c71e3e', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '67', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['em1']}}, 'uuid': '4C4C4544-0033-5310-8052-CAC04F4A354A', 'lastClientIface': 'rhevm', 'nics': {'em1': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': 'd4:ae:52:c7:1e:3e', 'DEVICE': 'em1', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3e', 'speed': 1000}, 'em2': {'addr': '', 'cfg': {'UUID': 'e32b7001-2514-471c-8c69-a075eed805e5', 'NM_CONTROLLED': 'yes', 'HWADDR': 'D4:AE:52:C7:1E:3F', 'BOOTPROTO': 'dhcp', 'DEVICE': 'em2', 'TYPE': 'Ethernet', 'ONBOOT': 'no'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3f', 'speed': 0}}, 'software_revision': '1', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:193293f409a', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '7842', 'cpuSpeed': '3301.000', 'numaNodes': {u'0': {'totalMemory': '7842', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-16::DEBUG::2014-10-23 17:12:43,470::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:43,550::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:43,551::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-17::DEBUG::2014-10-23 17:12:43,551::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >Thread-17::DEBUG::2014-10-23 17:12:43,583::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'PowerEdge R210 II', 'systemUUID': '4C4C4544-0033-5310-8052-CAC04F4A354A', 'systemSerialNumber': 'J3SRJ5J', 'systemManufacturer': 'Dell Inc.'} >Thread-17::DEBUG::2014-10-23 17:12:43,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:44,088::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:44,089::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-18::DEBUG::2014-10-23 17:12:44,089::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getCapabilities' in bridge with {} >Thread-18::DEBUG::2014-10-23 17:12:44,090::utils::738::root::(execCmd) /sbin/ip route show to 0.0.0.0/0 table all (cwd None) >Thread-18::DEBUG::2014-10-23 17:12:44,094::utils::758::root::(execCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-18::DEBUG::2014-10-23 17:12:44,114::caps::646::root::(get) VirtioRNG DISABLED: libvirt version 0.10.2-29.el6_5.12 required >= 0.10.2-31 >Thread-18::DEBUG::2014-10-23 17:12:44,115::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getCapabilities' in bridge with {'HBAInventory': {'iSCSI': [{'InitiatorName': 'iqn.1994-05.com.redhat:193293f409a'}], 'FC': []}, 'packages2': {'kernel': {'release': '431.el6.x86_64', 'buildtime': 1384118394.0, 'version': '2.6.32'}, 'spice-server': {'release': '6.el6_5.2', 'buildtime': 1401730350L, 'version': '0.12.4'}, 'vdsm': {'release': '1.el6ev', 'buildtime': 1413902785L, 'version': '4.16.7.1'}, 'qemu-kvm': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'libvirt': {'release': '29.el6_5.12', 'buildtime': 1408610144L, 'version': '0.10.2'}, 'qemu-img': {'release': '2.415.el6_5.14', 'buildtime': 1407344452L, 'version': '0.12.1.2'}, 'mom': {'release': '2.el6', 'buildtime': 1403794344L, 'version': '0.4.1'}}, 'numaNodeDistance': {'0': [10]}, 'cpuModel': 'Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz', 'liveMerge': 'false', 'hooks': {'after_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_network_setup': {'30_ethtool_options': {'md5': '2f6fe7f77eb498fb9493ef9ea5ad8705'}}, 'after_vm_destroy': {'50_vhostmd': {'md5': '92498ed80c0219749829ecce2813fc7c'}}, 'before_vm_start': {'50_qemucmdline': {'md5': 'a884929ad6f5eb039887157288867409'}, '50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}, '10_faqemu': {'md5': 'e3cb338be6b9075a04bed0e7400ab7e2'}, '50_macspoof': {'md5': '25deea559772719b90117f2e1009d823'}}, 'after_device_migrate_destination': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'after_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'ec28d5c17500ef0cf8f1a6393ac58506'}}, 'before_nic_hotplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}, 'before_vm_migrate_destination': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_nic_hotunplug': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_vm_dehibernate': {'50_vhostmd': {'md5': 'f3ee6dbf6fbd01333bd3e32afec4fbba'}}, 'after_device_destroy': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': '356db41e728db63bf6f7b28c5cfcbf70'}}, 'before_device_create': {'openstacknet_utils.py': {'md5': 'feebf44191d76fbc26e579a6323ce55c'}, '50_openstacknet': {'md5': 'af0f23c0d3e18c574385e4595f69e80b'}, '50_macspoof': {'md5': 'ca9c8ade572d4b5b4390755d4196c416'}}}, 'cpuSockets': '1', 'vmTypes': ['kvm'], 'selinux': {'mode': '1'}, 'kdumpStatus': 0, 'supportedProtocols': ['2.2', '2.3'], 'networks': {'rhevm': {'iface': 'rhevm', 'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'bridged': True, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'gateway': '10.34.63.254', 'bootproto4': 'dhcp', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'mtu': '1500', 'ipv6gateway': 'fe80:52:0:223c::3fe', 'ports': ['em1']}}, 'bridges': {'rhevm': {'addr': '10.34.62.206', 'cfg': {'DEFROUTE': 'yes', 'HOTPLUG': 'no', 'MTU': '1500', 'DELAY': '0', 'NM_CONTROLLED': 'no', 'BOOTPROTO': 'dhcp', 'STP': 'off', 'DEVICE': 'rhevm', 'TYPE': 'Bridge', 'ONBOOT': 'yes'}, 'ipv6addrs': ['2620:52:0:223c:d6ae:52ff:fec7:1e3e/64', 'fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '255.255.252.0', 'stp': 'off', 'ipv4addrs': ['10.34.62.206/22'], 'ipv6gateway': 'fe80:52:0:223c::3fe', 'gateway': '10.34.63.254', 'opts': {'topology_change_detected': '0', 'multicast_last_member_count': '2', 'hash_elasticity': '4', 'multicast_query_response_interval': '999', 'multicast_snooping': '1', 'multicast_startup_query_interval': '3124', 'hello_timer': '102', 'multicast_querier_interval': '25496', 'max_age': '1999', 'hash_max': '512', 'stp_state': '0', 'root_id': '8000.d4ae52c71e3e', 'priority': '32768', 'multicast_membership_interval': '25996', 'root_path_cost': '0', 'root_port': '0', 'multicast_querier': '0', 'multicast_startup_query_count': '2', 'hello_time': '199', 'topology_change': '0', 'bridge_id': '8000.d4ae52c71e3e', 'topology_change_timer': '0', 'ageing_time': '29995', 'gc_timer': '2', 'group_addr': '1:80:c2:0:0:0', 'tcn_timer': '0', 'multicast_query_interval': '12498', 'multicast_last_member_interval': '99', 'multicast_router': '1', 'forward_delay': '0'}, 'ports': ['em1']}}, 'uuid': '4C4C4544-0033-5310-8052-CAC04F4A354A', 'lastClientIface': 'rhevm', 'nics': {'em1': {'addr': '', 'cfg': {'BRIDGE': 'rhevm', 'NM_CONTROLLED': 'no', 'MTU': '1500', 'HWADDR': 'd4:ae:52:c7:1e:3e', 'DEVICE': 'em1', 'ONBOOT': 'yes'}, 'ipv6addrs': ['fe80::d6ae:52ff:fec7:1e3e/64'], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3e', 'speed': 1000}, 'em2': {'addr': '', 'cfg': {'UUID': 'e32b7001-2514-471c-8c69-a075eed805e5', 'NM_CONTROLLED': 'yes', 'HWADDR': 'D4:AE:52:C7:1E:3F', 'BOOTPROTO': 'dhcp', 'DEVICE': 'em2', 'TYPE': 'Ethernet', 'ONBOOT': 'no'}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': 'd4:ae:52:c7:1e:3f', 'speed': 0}}, 'software_revision': '1', 'clusterLevels': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'cpuFlags': u'fpu,vme,de,pse,tsc,msr,pae,mce,cx8,apic,sep,mtrr,pge,mca,cmov,pat,pse36,clflush,dts,acpi,mmx,fxsr,sse,sse2,ss,ht,tm,pbe,syscall,nx,rdtscp,lm,constant_tsc,arch_perfmon,pebs,bts,rep_good,xtopology,nonstop_tsc,aperfmperf,pni,pclmulqdq,dtes64,monitor,ds_cpl,vmx,smx,est,tm2,ssse3,cx16,xtpr,pdcm,pcid,sse4_1,sse4_2,x2apic,popcnt,tsc_deadline_timer,aes,xsave,avx,f16c,rdrand,lahf_lm,ida,arat,epb,xsaveopt,pln,pts,dts,tpr_shadow,vnmi,flexpriority,ept,vpid,fsgsbase,smep,erms,model_Nehalem,model_Conroe,model_coreduo,model_core2duo,model_Penryn,model_Westmere,model_n270,model_SandyBridge', 'ISCSIInitiatorName': 'iqn.1994-05.com.redhat:193293f409a', 'netConfigDirty': 'False', 'supportedENGINEs': ['3.0', '3.1', '3.2', '3.3', '3.4', '3.5'], 'autoNumaBalancing': 2, 'reservedMem': '321', 'bondings': {'bond4': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond0': {'addr': '', 'cfg': {}, 'ipv6addrs': [], 'mtu': '1500', 'netmask': '', 'ipv4addrs': [], 'hwaddr': '00:00:00:00:00:00', 'slaves': [], 'opts': {}}, 'bond1': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond2': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}, 'bond3': {'addr': '', 'cfg': {}, 'mtu': '1500', 'netmask': '', 'slaves': [], 'hwaddr': '00:00:00:00:00:00'}}, 'software_version': '4.16', 'memSize': '7842', 'cpuSpeed': '3301.000', 'numaNodes': {u'0': {'totalMemory': '7842', 'cpus': [0, 1, 2, 3, 4, 5, 6, 7]}}, 'version_name': 'Snow Man', 'vlans': {}, 'cpuCores': '4', 'kvmEnabled': 'true', 'guestOverhead': '65', 'supportedRHEVMs': ['3.0'], 'cpuThreads': '8', 'emulatedMachines': [u'rhel6.5.0', u'pc', u'rhel6.4.0', u'rhel6.3.0', u'rhel6.2.0', u'rhel6.1.0', u'rhel6.0.0', u'rhel5.5.0', u'rhel5.4.4', u'rhel5.4.0'], 'operatingSystem': {'release': '6.5.0.1.el6', 'version': '6Server', 'name': 'RHEL'}} >Thread-18::DEBUG::2014-10-23 17:12:44,118::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:44,185::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-19::DEBUG::2014-10-23 17:12:44,185::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getHardwareInfo' in bridge with {} >JsonRpcServer::DEBUG::2014-10-23 17:12:44,185::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-19::DEBUG::2014-10-23 17:12:44,187::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getHardwareInfo' in bridge with {'systemProductName': 'PowerEdge R210 II', 'systemUUID': '4C4C4544-0033-5310-8052-CAC04F4A354A', 'systemSerialNumber': 'J3SRJ5J', 'systemManufacturer': 'Dell Inc.'} >Thread-19::DEBUG::2014-10-23 17:12:44,188::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:44,258::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:44,259::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-20::DEBUG::2014-10-23 17:12:44,259::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:44,335::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:44,335::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-21::DEBUG::2014-10-23 17:12:44,335::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.setMOMPolicyParameters' in bridge with {u'key_value_store': {u'balloonEnabled': False, u'ksmEnabled': True}} >Thread-21::DEBUG::2014-10-23 17:12:44,388::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.setMOMPolicyParameters' in bridge with True >Thread-21::DEBUG::2014-10-23 17:12:44,388::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:47,697::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:47,698::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-22::DEBUG::2014-10-23 17:12:47,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6e5347d-44f7-42c2-ac3b-5609302d0f52`::moving from state init -> state preparing >Thread-22::INFO::2014-10-23 17:12:47,700::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-22::INFO::2014-10-23 17:12:47,700::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-22::DEBUG::2014-10-23 17:12:47,700::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6e5347d-44f7-42c2-ac3b-5609302d0f52`::finished: {} >Thread-22::DEBUG::2014-10-23 17:12:47,700::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6e5347d-44f7-42c2-ac3b-5609302d0f52`::moving from state preparing -> state finished >Thread-22::DEBUG::2014-10-23 17:12:47,700::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-22::DEBUG::2014-10-23 17:12:47,700::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-22::DEBUG::2014-10-23 17:12:47,700::task::993::Storage.TaskManager.Task::(_decref) Task=`b6e5347d-44f7-42c2-ac3b-5609302d0f52`::ref 0 aborting False >Thread-22::DEBUG::2014-10-23 17:12:47,702::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:47,788::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:47,789::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-23::DEBUG::2014-10-23 17:12:47,789::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:51,162::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:51,162::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-24::DEBUG::2014-10-23 17:12:51,163::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:54,202::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:54,202::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-25::DEBUG::2014-10-23 17:12:54,203::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:12:57,242::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:12:57,243::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-26::DEBUG::2014-10-23 17:12:57,244::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:00,275::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:00,275::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-27::DEBUG::2014-10-23 17:13:00,276::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:03,307::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:03,308::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-28::DEBUG::2014-10-23 17:13:03,309::task::595::Storage.TaskManager.Task::(_updateState) Task=`76c75598-297d-418a-abc0-0ce73fa811aa`::moving from state init -> state preparing >Thread-28::INFO::2014-10-23 17:13:03,309::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-28::INFO::2014-10-23 17:13:03,309::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-28::DEBUG::2014-10-23 17:13:03,309::task::1191::Storage.TaskManager.Task::(prepare) Task=`76c75598-297d-418a-abc0-0ce73fa811aa`::finished: {} >Thread-28::DEBUG::2014-10-23 17:13:03,310::task::595::Storage.TaskManager.Task::(_updateState) Task=`76c75598-297d-418a-abc0-0ce73fa811aa`::moving from state preparing -> state finished >Thread-28::DEBUG::2014-10-23 17:13:03,310::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-28::DEBUG::2014-10-23 17:13:03,310::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-28::DEBUG::2014-10-23 17:13:03,310::task::993::Storage.TaskManager.Task::(_decref) Task=`76c75598-297d-418a-abc0-0ce73fa811aa`::ref 0 aborting False >Thread-28::DEBUG::2014-10-23 17:13:03,311::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:03,359::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:03,359::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-29::DEBUG::2014-10-23 17:13:03,360::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:06,530::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:06,531::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-30::DEBUG::2014-10-23 17:13:06,532::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:09,564::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:09,565::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-31::DEBUG::2014-10-23 17:13:09,566::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:12,582::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:12,583::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-32::DEBUG::2014-10-23 17:13:12,584::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:15,610::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:15,611::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-33::DEBUG::2014-10-23 17:13:15,611::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:18,665::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:18,666::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-34::DEBUG::2014-10-23 17:13:18,667::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd8aceb-701e-44d7-b801-7e4f39fff260`::moving from state init -> state preparing >Thread-34::INFO::2014-10-23 17:13:18,668::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-34::INFO::2014-10-23 17:13:18,668::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-34::DEBUG::2014-10-23 17:13:18,668::task::1191::Storage.TaskManager.Task::(prepare) Task=`bdd8aceb-701e-44d7-b801-7e4f39fff260`::finished: {} >Thread-34::DEBUG::2014-10-23 17:13:18,668::task::595::Storage.TaskManager.Task::(_updateState) Task=`bdd8aceb-701e-44d7-b801-7e4f39fff260`::moving from state preparing -> state finished >Thread-34::DEBUG::2014-10-23 17:13:18,668::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-34::DEBUG::2014-10-23 17:13:18,668::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-34::DEBUG::2014-10-23 17:13:18,668::task::993::Storage.TaskManager.Task::(_decref) Task=`bdd8aceb-701e-44d7-b801-7e4f39fff260`::ref 0 aborting False >Thread-34::DEBUG::2014-10-23 17:13:18,670::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:18,724::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:18,724::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-35::DEBUG::2014-10-23 17:13:18,725::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:21,674::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-36::DEBUG::2014-10-23 17:13:21,674::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getDeviceList' in bridge with {u'storageType': 3} >JsonRpcServer::DEBUG::2014-10-23 17:13:21,674::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-36::DEBUG::2014-10-23 17:13:21,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`3a7c28a5-c5fa-4d86-93ba-43c4f367ef2a`::moving from state init -> state preparing >Thread-36::INFO::2014-10-23 17:13:21,675::logUtils::44::dispatcher::(wrapper) Run and protect: getDeviceList(storageType=3, options={}) >Thread-36::DEBUG::2014-10-23 17:13:21,675::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-36::DEBUG::2014-10-23 17:13:21,676::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-36::DEBUG::2014-10-23 17:13:21,676::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-36::DEBUG::2014-10-23 17:13:21,676::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-36::DEBUG::2014-10-23 17:13:21,676::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-36::DEBUG::2014-10-23 17:13:21,676::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-36::DEBUG::2014-10-23 17:13:21,684::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-36::DEBUG::2014-10-23 17:13:21,684::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-36::DEBUG::2014-10-23 17:13:21,781::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-36::DEBUG::2014-10-23 17:13:21,781::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,782::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-36::DEBUG::2014-10-23 17:13:21,782::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,783::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >Thread-36::DEBUG::2014-10-23 17:13:21,876::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-36::DEBUG::2014-10-23 17:13:21,877::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-36::DEBUG::2014-10-23 17:13:21,891::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --test --metadatasize 128m --metadatacopies 2 --metadataignore y (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:21,906::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:21,910::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-37::DEBUG::2014-10-23 17:13:21,911::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-36::DEBUG::2014-10-23 17:13:21,977::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = " TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.\n Please enter a physical volume path\n Run `pvcreate --help' for more information.\n"; <rc> = 3 >Thread-36::DEBUG::2014-10-23 17:13:21,977::lvm::852::Storage.LVM::(testPVCreate) rc: 3, out: [], err: [' TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.', ' Please enter a physical volume path', " Run `pvcreate --help' for more information."], unusedDevs: set([]), usedDevs: set([]) >Thread-36::INFO::2014-10-23 17:13:21,977::logUtils::47::dispatcher::(wrapper) Run and protect: getDeviceList, Return response: {'devList': []} >Thread-36::DEBUG::2014-10-23 17:13:21,977::task::1191::Storage.TaskManager.Task::(prepare) Task=`3a7c28a5-c5fa-4d86-93ba-43c4f367ef2a`::finished: {'devList': []} >Thread-36::DEBUG::2014-10-23 17:13:21,978::task::595::Storage.TaskManager.Task::(_updateState) Task=`3a7c28a5-c5fa-4d86-93ba-43c4f367ef2a`::moving from state preparing -> state finished >Thread-36::DEBUG::2014-10-23 17:13:21,978::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-36::DEBUG::2014-10-23 17:13:21,978::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-36::DEBUG::2014-10-23 17:13:21,978::task::993::Storage.TaskManager.Task::(_decref) Task=`3a7c28a5-c5fa-4d86-93ba-43c4f367ef2a`::ref 0 aborting False >Thread-36::DEBUG::2014-10-23 17:13:21,978::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getDeviceList' in bridge with [] >Thread-36::DEBUG::2014-10-23 17:13:21,978::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:24,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:24,939::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-38::DEBUG::2014-10-23 17:13:24,940::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:27,962::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:27,963::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-39::DEBUG::2014-10-23 17:13:27,966::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:30,993::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:30,993::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-40::DEBUG::2014-10-23 17:13:30,994::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:33,851::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-41::DEBUG::2014-10-23 17:13:33,852::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'ISCSIConnection.discoverSendTargets' in bridge with {u'host': u'10.34.63.202', u'port': u'3260'} >JsonRpcServer::DEBUG::2014-10-23 17:13:33,852::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-41::DEBUG::2014-10-23 17:13:33,853::task::595::Storage.TaskManager.Task::(_updateState) Task=`46e6d944-9f33-4a94-b7fc-9d250a10fd85`::moving from state init -> state preparing >Thread-41::INFO::2014-10-23 17:13:33,853::logUtils::44::dispatcher::(wrapper) Run and protect: discoverSendTargets(con={'connection': u'10.34.63.202', 'password': '******', 'port': u'3260', 'user': ''}, options=None) >Thread-41::DEBUG::2014-10-23 17:13:33,853::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.34.63.202:3260 --op=new (cwd None) >Thread-41::DEBUG::2014-10-23 17:13:33,863::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-41::DEBUG::2014-10-23 17:13:33,864::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.34.63.202:3260 --discover (cwd None) >Thread-41::DEBUG::2014-10-23 17:13:33,879::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-41::DEBUG::2014-10-23 17:13:33,880::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m discoverydb -t sendtargets -I default -p 10.34.63.202:3260 --op=delete (cwd None) >Thread-41::DEBUG::2014-10-23 17:13:33,889::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-41::INFO::2014-10-23 17:13:33,889::logUtils::47::dispatcher::(wrapper) Run and protect: discoverSendTargets, Return response: {'fullTargets': ['10.34.63.202:3260,1 iqn.2011-01.brq.str-01:pstehlik', '10.34.63.202:3260,1 iqn.2012-06.brq.str-01:brq-setup', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:cmestreg', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:kjachim', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:lbednar', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:mbetak', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:ha-tests', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:jbelka', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:mpavlik'], 'targets': ['iqn.2011-01.brq.str-01:pstehlik', 'iqn.2012-06.brq.str-01:brq-setup', 'iqn.2012-10.brq.str-01:cmestreg', 'iqn.2012-10.brq.str-01:kjachim', 'iqn.2012-10.brq.str-01:lbednar', 'iqn.2012-10.brq.str-01:mbetak', 'iqn.2013-12.brq.str-01:ha-tests', 'iqn.2013-12.brq.str-01:jbelka', 'iqn.2013-12.brq.str-01:mpavlik']} >Thread-41::DEBUG::2014-10-23 17:13:33,890::task::1191::Storage.TaskManager.Task::(prepare) Task=`46e6d944-9f33-4a94-b7fc-9d250a10fd85`::finished: {'fullTargets': ['10.34.63.202:3260,1 iqn.2011-01.brq.str-01:pstehlik', '10.34.63.202:3260,1 iqn.2012-06.brq.str-01:brq-setup', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:cmestreg', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:kjachim', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:lbednar', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:mbetak', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:ha-tests', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:jbelka', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:mpavlik'], 'targets': ['iqn.2011-01.brq.str-01:pstehlik', 'iqn.2012-06.brq.str-01:brq-setup', 'iqn.2012-10.brq.str-01:cmestreg', 'iqn.2012-10.brq.str-01:kjachim', 'iqn.2012-10.brq.str-01:lbednar', 'iqn.2012-10.brq.str-01:mbetak', 'iqn.2013-12.brq.str-01:ha-tests', 'iqn.2013-12.brq.str-01:jbelka', 'iqn.2013-12.brq.str-01:mpavlik']} >Thread-41::DEBUG::2014-10-23 17:13:33,890::task::595::Storage.TaskManager.Task::(_updateState) Task=`46e6d944-9f33-4a94-b7fc-9d250a10fd85`::moving from state preparing -> state finished >Thread-41::DEBUG::2014-10-23 17:13:33,890::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-41::DEBUG::2014-10-23 17:13:33,890::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-41::DEBUG::2014-10-23 17:13:33,890::task::993::Storage.TaskManager.Task::(_decref) Task=`46e6d944-9f33-4a94-b7fc-9d250a10fd85`::ref 0 aborting False >Thread-41::DEBUG::2014-10-23 17:13:33,890::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'ISCSIConnection.discoverSendTargets' in bridge with ['10.34.63.202:3260,1 iqn.2011-01.brq.str-01:pstehlik', '10.34.63.202:3260,1 iqn.2012-06.brq.str-01:brq-setup', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:cmestreg', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:kjachim', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:lbednar', '10.34.63.202:3260,1 iqn.2012-10.brq.str-01:mbetak', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:ha-tests', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:jbelka', '10.34.63.202:3260,1 iqn.2013-12.brq.str-01:mpavlik'] >Thread-41::DEBUG::2014-10-23 17:13:33,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:34,081::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:34,081::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-42::DEBUG::2014-10-23 17:13:34,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`060f1ea2-c64f-485f-a693-fef3683a5990`::moving from state init -> state preparing >Thread-42::INFO::2014-10-23 17:13:34,083::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-42::INFO::2014-10-23 17:13:34,083::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-42::DEBUG::2014-10-23 17:13:34,083::task::1191::Storage.TaskManager.Task::(prepare) Task=`060f1ea2-c64f-485f-a693-fef3683a5990`::finished: {} >Thread-42::DEBUG::2014-10-23 17:13:34,083::task::595::Storage.TaskManager.Task::(_updateState) Task=`060f1ea2-c64f-485f-a693-fef3683a5990`::moving from state preparing -> state finished >Thread-42::DEBUG::2014-10-23 17:13:34,083::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-42::DEBUG::2014-10-23 17:13:34,083::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-42::DEBUG::2014-10-23 17:13:34,083::task::993::Storage.TaskManager.Task::(_decref) Task=`060f1ea2-c64f-485f-a693-fef3683a5990`::ref 0 aborting False >Thread-42::DEBUG::2014-10-23 17:13:34,085::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:34,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:34,149::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-43::DEBUG::2014-10-23 17:13:34,149::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:36,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-44::DEBUG::2014-10-23 17:13:36,301::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'00000000-0000-0000-0000-000000000000', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 3} >JsonRpcServer::DEBUG::2014-10-23 17:13:36,301::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-44::DEBUG::2014-10-23 17:13:36,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c50fb62-7317-4fb6-b945-dc3ea1f23cb7`::moving from state init -> state preparing >Thread-44::INFO::2014-10-23 17:13:36,302::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'00000000-0000-0000-0000-000000000000', u'port': u'3260'}], options=None) >Thread-44::DEBUG::2014-10-23 17:13:36,302::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-44::DEBUG::2014-10-23 17:13:36,311::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-44::DEBUG::2014-10-23 17:13:36,311::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-44::DEBUG::2014-10-23 17:13:36,847::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-44::DEBUG::2014-10-23 17:13:36,848::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-44::DEBUG::2014-10-23 17:13:36,858::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-44::DEBUG::2014-10-23 17:13:36,858::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-44::DEBUG::2014-10-23 17:13:36,858::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-44::DEBUG::2014-10-23 17:13:36,961::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' No volume groups found\n'; <rc> = 0 >Thread-44::DEBUG::2014-10-23 17:13:36,961::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-44::DEBUG::2014-10-23 17:13:36,961::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: () >Thread-44::DEBUG::2014-10-23 17:13:36,962::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {} >Thread-44::INFO::2014-10-23 17:13:36,962::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} >Thread-44::DEBUG::2014-10-23 17:13:36,962::task::1191::Storage.TaskManager.Task::(prepare) Task=`9c50fb62-7317-4fb6-b945-dc3ea1f23cb7`::finished: {'statuslist': [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}]} >Thread-44::DEBUG::2014-10-23 17:13:36,962::task::595::Storage.TaskManager.Task::(_updateState) Task=`9c50fb62-7317-4fb6-b945-dc3ea1f23cb7`::moving from state preparing -> state finished >Thread-44::DEBUG::2014-10-23 17:13:36,962::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-44::DEBUG::2014-10-23 17:13:36,962::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-44::DEBUG::2014-10-23 17:13:36,962::task::993::Storage.TaskManager.Task::(_decref) Task=`9c50fb62-7317-4fb6-b945-dc3ea1f23cb7`::ref 0 aborting False >Thread-44::DEBUG::2014-10-23 17:13:36,962::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'00000000-0000-0000-0000-000000000000'}] >Thread-44::DEBUG::2014-10-23 17:13:36,963::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:37,088::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-45::DEBUG::2014-10-23 17:13:37,089::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getDeviceList' in bridge with {u'storageType': 3} >JsonRpcServer::DEBUG::2014-10-23 17:13:37,089::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-45::DEBUG::2014-10-23 17:13:37,091::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7b8f1bf-e9f2-4821-9460-c075b17d443a`::moving from state init -> state preparing >Thread-45::INFO::2014-10-23 17:13:37,091::logUtils::44::dispatcher::(wrapper) Run and protect: getDeviceList(storageType=3, options={}) >Thread-45::DEBUG::2014-10-23 17:13:37,091::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-45::DEBUG::2014-10-23 17:13:37,091::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-45::DEBUG::2014-10-23 17:13:37,091::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-45::DEBUG::2014-10-23 17:13:37,091::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-45::DEBUG::2014-10-23 17:13:37,091::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-45::DEBUG::2014-10-23 17:13:37,091::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-45::DEBUG::2014-10-23 17:13:37,109::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-45::DEBUG::2014-10-23 17:13:37,110::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-45::DEBUG::2014-10-23 17:13:37,221::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-45::DEBUG::2014-10-23 17:13:37,221::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,221::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,222::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,222::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,222::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,222::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,222::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-45::DEBUG::2014-10-23 17:13:37,222::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,223::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:37,277::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:37,278::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-46::DEBUG::2014-10-23 17:13:37,279::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-45::DEBUG::2014-10-23 17:13:37,334::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-45::DEBUG::2014-10-23 17:13:37,334::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-45::DEBUG::2014-10-23 17:13:37,373::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --test --metadatasize 128m --metadatacopies 2 --metadataignore y /dev/mapper/1cmg02 /dev/mapper/1cmg01 (cwd None) >Thread-45::DEBUG::2014-10-23 17:13:37,489::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.\n Can\'t initialize physical volume "/dev/mapper/1cmg01" of volume group "2d4b6e8f-878f-4d9e-96f5-176b679b43e4" without -ff\n'; <rc> = 5 >Thread-45::DEBUG::2014-10-23 17:13:37,491::lvm::852::Storage.LVM::(testPVCreate) rc: 5, out: [' Physical volume "/dev/mapper/1cmg02" successfully created'], err: [' TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.', ' Can\'t initialize physical volume "/dev/mapper/1cmg01" of volume group "2d4b6e8f-878f-4d9e-96f5-176b679b43e4" without -ff'], unusedDevs: set(['/dev/mapper/1cmg02']), usedDevs: set(['/dev/mapper/1cmg01']) >Thread-45::INFO::2014-10-23 17:13:37,491::logUtils::47::dispatcher::(wrapper) Run and protect: getDeviceList, Return response: {'devList': [{'status': 'free', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'initiatorname': 'default', 'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK'}, {'status': 'used', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'fwrev': '0001', 'vgUUID': 'bzAFwQ-vOk7-MphX-LMdE-xAvd-CwDu-04ovkt', 'pathlist': [{'initiatorname': 'default', 'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': 'ooeF1z-kQE6-NTps-jaDa-AukH-DbmK-4XbleL', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK'}]} >Thread-45::DEBUG::2014-10-23 17:13:37,496::task::1191::Storage.TaskManager.Task::(prepare) Task=`a7b8f1bf-e9f2-4821-9460-c075b17d443a`::finished: {'devList': [{'status': 'free', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ'}, {'status': 'used', 'fwrev': '0001', 'vgUUID': 'bzAFwQ-vOk7-MphX-LMdE-xAvd-CwDu-04ovkt', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'pvUUID': 'ooeF1z-kQE6-NTps-jaDa-AukH-DbmK-4XbleL'}]} >Thread-45::DEBUG::2014-10-23 17:13:37,496::task::595::Storage.TaskManager.Task::(_updateState) Task=`a7b8f1bf-e9f2-4821-9460-c075b17d443a`::moving from state preparing -> state finished >Thread-45::DEBUG::2014-10-23 17:13:37,496::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-45::DEBUG::2014-10-23 17:13:37,497::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-45::DEBUG::2014-10-23 17:13:37,497::task::993::Storage.TaskManager.Task::(_decref) Task=`a7b8f1bf-e9f2-4821-9460-c075b17d443a`::ref 0 aborting False >Thread-45::WARNING::2014-10-23 17:13:37,497::Bridge::212::root::(_typeFixup) Fixing up type BlockDevicePathInfo >Thread-45::WARNING::2014-10-23 17:13:37,500::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-45::WARNING::2014-10-23 17:13:37,500::Bridge::212::root::(_typeFixup) Fixing up type BlockDevicePathInfo >Thread-45::WARNING::2014-10-23 17:13:37,500::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-45::DEBUG::2014-10-23 17:13:37,500::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getDeviceList' in bridge with [{'status': 'free', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'pathstatus': [{'deviceType': 'iSCSI', 'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ'}, {'status': 'used', 'fwrev': '0001', 'vgUUID': 'bzAFwQ-vOk7-MphX-LMdE-xAvd-CwDu-04ovkt', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'pathstatus': [{'deviceType': 'iSCSI', 'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'pvUUID': 'ooeF1z-kQE6-NTps-jaDa-AukH-DbmK-4XbleL'}] >Thread-45::DEBUG::2014-10-23 17:13:37,501::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:40,305::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:40,306::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-47::DEBUG::2014-10-23 17:13:40,307::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:43,333::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:43,333::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-48::DEBUG::2014-10-23 17:13:43,335::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:44,787::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-49::DEBUG::2014-10-23 17:13:44,787::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.create' in bridge with {u'devlist': [u'1cmg01'], u'force': True, u'name': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'} >JsonRpcServer::DEBUG::2014-10-23 17:13:44,787::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-49::DEBUG::2014-10-23 17:13:44,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`75ecf2d1-87bc-473f-888f-32cc396bc1fd`::moving from state init -> state preparing >Thread-49::INFO::2014-10-23 17:13:44,788::logUtils::44::dispatcher::(wrapper) Run and protect: createVG(vgname=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', devlist=[u'1cmg01'], force=True, options=None) >Thread-49::DEBUG::2014-10-23 17:13:44,789::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' -y -ff --metadatasize 128m --metadatacopies 2 --metadataignore y /dev/mapper/1cmg01 (cwd None) >Thread-49::DEBUG::2014-10-23 17:13:44,907::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: Forcing physical volume creation on /dev/mapper/1cmg01 of volume group "2d4b6e8f-878f-4d9e-96f5-176b679b43e4"\n'; <rc> = 0 >Thread-49::DEBUG::2014-10-23 17:13:44,908::lvm::483::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:44,908::lvm::486::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:44,908::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --metadataignore n /dev/mapper/1cmg01 (cwd None) >Thread-49::DEBUG::2014-10-23 17:13:45,022::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49::DEBUG::2014-10-23 17:13:45,023::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --physicalextentsize 128m --addtag RHAT_storage_domain_UNREADY 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f /dev/mapper/1cmg01 (cwd None) >Thread-49::DEBUG::2014-10-23 17:13:45,180::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49::DEBUG::2014-10-23 17:13:45,180::lvm::483::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:45,180::lvm::486::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:45,180::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:45,180::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:45,181::lvm::923::Storage.LVM::(createVG) Cache after createvg {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': Stub(name=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', stale=True)} >Thread-49::DEBUG::2014-10-23 17:13:45,181::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-49::DEBUG::2014-10-23 17:13:45,181::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-49::DEBUG::2014-10-23 17:13:45,275::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-49::DEBUG::2014-10-23 17:13:45,275::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-49::INFO::2014-10-23 17:13:45,276::logUtils::47::dispatcher::(wrapper) Run and protect: createVG, Return response: {'uuid': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP'} >Thread-49::DEBUG::2014-10-23 17:13:45,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`75ecf2d1-87bc-473f-888f-32cc396bc1fd`::finished: {'uuid': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP'} >Thread-49::DEBUG::2014-10-23 17:13:45,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`75ecf2d1-87bc-473f-888f-32cc396bc1fd`::moving from state preparing -> state finished >Thread-49::DEBUG::2014-10-23 17:13:45,276::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-49::DEBUG::2014-10-23 17:13:45,276::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-49::DEBUG::2014-10-23 17:13:45,276::task::993::Storage.TaskManager.Task::(_decref) Task=`75ecf2d1-87bc-473f-888f-32cc396bc1fd`::ref 0 aborting False >Thread-49::DEBUG::2014-10-23 17:13:45,276::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.create' in bridge with L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP >Thread-49::DEBUG::2014-10-23 17:13:45,277::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:45,294::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:45,294::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-50::DEBUG::2014-10-23 17:13:45,294::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.create' in bridge with {u'name': u'iscsi', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', u'version': u'3', u'storagedomainID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'} >Thread-50::DEBUG::2014-10-23 17:13:45,296::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7e72403-82e9-400b-8e65-7b430850b72f`::moving from state init -> state preparing >Thread-50::INFO::2014-10-23 17:13:45,296::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=3, sdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', domainName=u'iscsi', typeSpecificArg=u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', domClass=1, domVersion=u'3', options=None) >Thread-50::ERROR::2014-10-23 17:13:45,296::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-50::ERROR::2014-10-23 17:13:45,296::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-50::ERROR::2014-10-23 17:13:45,297::sdc::143::Storage.StorageDomainCache::(_findDomain) domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: (u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f',) >Thread-50::INFO::2014-10-23 17:13:45,298::blockSD::484::Storage.StorageDomain::(create) sdUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f domainName=iscsi domClass=1 vgUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP storageType=3 version=3 >Thread-50::DEBUG::2014-10-23 17:13:45,298::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,298::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,394::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,394::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,394::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,394::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,485::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,485::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-50::DEBUG::2014-10-23 17:13:45,485::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-50::INFO::2014-10-23 17:13:45,485::blockSD::468::Storage.StorageDomain::(metaSize) size 512 MB (metaratio 262144) >Thread-50::DEBUG::2014-10-23 17:13:45,486::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 512m --name metadata 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,625::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,625::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,625::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,625::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,625::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,626::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,626::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,718::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,718::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-50::DEBUG::2014-10-23 17:13:45,718::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,719::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,719::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,811::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,811::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::INFO::2014-10-23 17:13:45,811::blockSD::653::Storage.StorageDomain::(getMetaDataMapping) Create: SORT MAPPING: ['/dev/mapper/1cmg01'] >Thread-50::DEBUG::2014-10-23 17:13:45,811::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,811::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size /dev/mapper/1cmg01 (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:45,908::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:45,908::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:45,909::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 16m --name outbox 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,039::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,040::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,040::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,040::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,040::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,040::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 2048m --name leases 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,155::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,155::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,155::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,155::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,155::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,156::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/leases (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,163::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,164::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 8m --name ids 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,278::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,278::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,278::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,279::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,279::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,279::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/ids (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,287::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,287::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 16m --name inbox 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:46,351::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:46,352::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-51::DEBUG::2014-10-23 17:13:46,354::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-50::DEBUG::2014-10-23 17:13:46,401::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,401::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,401::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,401::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,401::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,402::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 1024m --name master 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,526::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,526::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,526::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,526::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,526::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:46,527::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:46,534::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:46,535::blockSD::1342::Storage.Misc.excCmd::(_createVMSfs) /usr/bin/sudo -n /sbin/mkfs -q -j -E nodiscard /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:47,394::blockSD::1342::Storage.Misc.excCmd::(_createVMSfs) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:47,395::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available n 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:47,494::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:47,494::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:47,495::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:47,495::blockSD::535::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata bs=1048576 seek=0 skip=0 conv=notrunc count=40 oflag=direct (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:47,896::blockSD::535::Storage.Misc.excCmd::(create) SUCCESS: <err> = '40+0 records in\n40+0 records out\n41943040 bytes (42 MB) copied, 0.397078 s, 106 MB/s\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:47,897::misc::262::Storage.Misc::(validateDDBytes) err: ['40+0 records in', '40+0 records out', '41943040 bytes (42 MB) copied, 0.397078 s, 106 MB/s'], size: 41943040 >Thread-50::DEBUG::2014-10-23 17:13:47,897::blockSD::537::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/inbox bs=1024000 seek=0 skip=0 conv=notrunc count=1 oflag=direct (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:47,911::blockSD::537::Storage.Misc.excCmd::(create) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00988704 s, 104 MB/s\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:47,911::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '1024000 bytes (1.0 MB) copied, 0.00988704 s, 104 MB/s'], size: 1024000 >Thread-50::DEBUG::2014-10-23 17:13:47,912::blockSD::539::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/outbox bs=1024000 seek=0 skip=0 conv=notrunc count=1 oflag=direct (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:47,926::blockSD::539::Storage.Misc.excCmd::(create) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00985395 s, 104 MB/s\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:47,926::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '1024000 bytes (1.0 MB) copied, 0.00985395 s, 104 MB/s'], size: 1024000 >Thread-50::DEBUG::2014-10-23 17:13:47,926::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-50::DEBUG::2014-10-23 17:13:47,927::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:47,927::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,022::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,022::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,023::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,116::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,116::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,116::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[] >Thread-50::DEBUG::2014-10-23 17:13:48,117::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-50::DEBUG::2014-10-23 17:13:48,117::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-50::DEBUG::2014-10-23 17:13:48,117::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-50::DEBUG::2014-10-23 17:13:48,117::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', u'DESCRIPTION=iscsi', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'PHYBLKSIZE=512', 'POOL_UUID=', 'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'ROLE=Regular', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-50::DEBUG::2014-10-23 17:13:48,117::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,117::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,117::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,117::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,118::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,118::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,209::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,209::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,210::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_PHYBLKSIZE=512, MDT_PV0=pv:1cmg01&44&uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC&44&pestart:0&44&pecount:677&44&mapoffset:0, MDT_SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f, MDT_CLASS=Data, MDT_LEASERETRIES=3, MDT_POOL_UUID=, MDT_IOOPTIMEOUTSEC=10, MDT_LOCKRENEWALINTERVALSEC=5, MDT_LOGBLKSIZE=512, MDT_LEASETIMESEC=60, MDT_VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP, MDT_VERSION=3, MDT_DESCRIPTION=iscsi, MDT_LOCKPOLICY=, MDT_TYPE=ISCSI, MDT_ROLE=Regular, MDT__SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572 removing= >Thread-50::DEBUG::2014-10-23 17:13:48,210::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --addtag MDT_LEASETIMESEC=60 --addtag 'MDT_PV0=pv:1cmg01&44&uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC&44&pestart:0&44&pecount:677&44&mapoffset:0' --addtag MDT_POOL_UUID= --addtag MDT_VERSION=3 --addtag MDT_TYPE=ISCSI --addtag MDT_DESCRIPTION=iscsi --addtag MDT_SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f --addtag MDT_LOGBLKSIZE=512 --addtag MDT_LEASERETRIES=3 --addtag MDT_IOOPTIMEOUTSEC=10 --addtag MDT_LOCKRENEWALINTERVALSEC=5 --addtag MDT_VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP --addtag MDT_PHYBLKSIZE=512 --addtag MDT_CLASS=Data --addtag MDT_LOCKPOLICY= --addtag MDT_ROLE=Regular --addtag MDT__SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,382::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,382::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,383::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,383::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-50::DEBUG::2014-10-23 17:13:48,383::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag RHAT_storage_domain_UNREADY --addtag RHAT_storage_domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,558::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,558::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,558::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,558::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-50::DEBUG::2014-10-23 17:13:48,559::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,563::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000510968 s, 4.0 MB/s\n'; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,563::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000510968 s, 4.0 MB/s'], size: 2048 >Thread-50::DEBUG::2014-10-23 17:13:48,563::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-50::DEBUG::2014-10-23 17:13:48,563::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-50::DEBUG::2014-10-23 17:13:48,564::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,564::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,655::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,656::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,656::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-50::DEBUG::2014-10-23 17:13:48,656::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-50::DEBUG::2014-10-23 17:13:48,779::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-50::DEBUG::2014-10-23 17:13:48,779::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,779::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-50::DEBUG::2014-10-23 17:13:48,780::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS' >Thread-50::DEBUG::2014-10-23 17:13:48,780::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS' >Thread-50::DEBUG::2014-10-23 17:13:48,780::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS' >Thread-50::DEBUG::2014-10-23 17:13:48,780::clusterlock::173::Storage.initSANLock::(initSANLock) Initializing SANLock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-50::DEBUG::2014-10-23 17:13:49,109::sd::443::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-50::DEBUG::2014-10-23 17:13:49,109::hsm::2691::Storage.HSM::(createStorageDomain) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain} >Thread-50::INFO::2014-10-23 17:13:49,109::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-50::DEBUG::2014-10-23 17:13:49,110::task::1191::Storage.TaskManager.Task::(prepare) Task=`b7e72403-82e9-400b-8e65-7b430850b72f`::finished: None >Thread-50::DEBUG::2014-10-23 17:13:49,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`b7e72403-82e9-400b-8e65-7b430850b72f`::moving from state preparing -> state finished >Thread-50::DEBUG::2014-10-23 17:13:49,110::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-50::DEBUG::2014-10-23 17:13:49,110::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-50::DEBUG::2014-10-23 17:13:49,110::task::993::Storage.TaskManager.Task::(_decref) Task=`b7e72403-82e9-400b-8e65-7b430850b72f`::ref 0 aborting False >Thread-50::DEBUG::2014-10-23 17:13:49,110::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.create' in bridge with True >Thread-50::DEBUG::2014-10-23 17:13:49,110::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:49,119::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:49,120::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-52::DEBUG::2014-10-23 17:13:49,121::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::moving from state init -> state preparing >Thread-52::INFO::2014-10-23 17:13:49,121::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', options=None) >Thread-52::DEBUG::2014-10-23 17:13:49,122::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`5ccae1c4-e8bf-49c2-9e3f-d99a1d4e48dd`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2830' at 'getStorageDomainStats' >Thread-52::DEBUG::2014-10-23 17:13:49,122::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'shared' >Thread-52::DEBUG::2014-10-23 17:13:49,122::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'shared' (1 active user) >Thread-52::DEBUG::2014-10-23 17:13:49,122::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`5ccae1c4-e8bf-49c2-9e3f-d99a1d4e48dd`::Granted request >Thread-52::DEBUG::2014-10-23 17:13:49,122::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::_resourcesAcquired: Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (shared) >Thread-52::DEBUG::2014-10-23 17:13:49,122::task::993::Storage.TaskManager.Task::(_decref) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::ref 1 aborting False >Thread-52::DEBUG::2014-10-23 17:13:49,122::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-52::DEBUG::2014-10-23 17:13:49,123::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-52::DEBUG::2014-10-23 17:13:49,123::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/metadata >Thread-52::DEBUG::2014-10-23 17:13:49,123::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/leases to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/leases >Thread-52::DEBUG::2014-10-23 17:13:49,123::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/ids to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/ids >Thread-52::DEBUG::2014-10-23 17:13:49,123::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/inbox to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/inbox >Thread-52::DEBUG::2014-10-23 17:13:49,124::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/outbox to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/outbox >Thread-52::DEBUG::2014-10-23 17:13:49,124::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master to /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md/master >Thread-52::DEBUG::2014-10-23 17:13:49,124::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-52::DEBUG::2014-10-23 17:13:49,124::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-52::DEBUG::2014-10-23 17:13:49,124::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-52::DEBUG::2014-10-23 17:13:49,124::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-52::DEBUG::2014-10-23 17:13:49,124::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-52::DEBUG::2014-10-23 17:13:49,125::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-52::DEBUG::2014-10-23 17:13:49,129::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000341287 s, 6.0 MB/s\n'; <rc> = 0 >Thread-52::DEBUG::2014-10-23 17:13:49,129::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000341287 s, 6.0 MB/s'], size: 2048 >Thread-52::DEBUG::2014-10-23 17:13:49,129::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-52::DEBUG::2014-10-23 17:13:49,129::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-52::DEBUG::2014-10-23 17:13:49,130::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-52::DEBUG::2014-10-23 17:13:49,130::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-52::DEBUG::2014-10-23 17:13:49,130::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-52::DEBUG::2014-10-23 17:13:49,222::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-52::DEBUG::2014-10-23 17:13:49,223::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-52::INFO::2014-10-23 17:13:49,223::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '86704652288', 'disktotal': '90865401856', 'mdafree': '67104768'}} >Thread-52::DEBUG::2014-10-23 17:13:49,223::task::1191::Storage.TaskManager.Task::(prepare) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::finished: {'stats': {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '86704652288', 'disktotal': '90865401856', 'mdafree': '67104768'}} >Thread-52::DEBUG::2014-10-23 17:13:49,223::task::595::Storage.TaskManager.Task::(_updateState) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::moving from state preparing -> state finished >Thread-52::DEBUG::2014-10-23 17:13:49,223::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': < ResourceRef 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', isValid: 'True' obj: 'None'>} >Thread-52::DEBUG::2014-10-23 17:13:49,223::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-52::DEBUG::2014-10-23 17:13:49,223::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >Thread-52::DEBUG::2014-10-23 17:13:49,224::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >Thread-52::DEBUG::2014-10-23 17:13:49,224::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >Thread-52::DEBUG::2014-10-23 17:13:49,224::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >Thread-52::DEBUG::2014-10-23 17:13:49,224::task::993::Storage.TaskManager.Task::(_decref) Task=`fc201f18-80c6-478b-a3d2-fea9acca6f6d`::ref 0 aborting False >Thread-52::DEBUG::2014-10-23 17:13:49,224::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:49,303::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:49,304::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-53::DEBUG::2014-10-23 17:13:49,304::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.getInfo' in bridge with {u'lvmvolumegroupID': u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP'} >Thread-53::DEBUG::2014-10-23 17:13:49,305::task::595::Storage.TaskManager.Task::(_updateState) Task=`a97f7495-3109-4f2c-a2cc-40446511b0a4`::moving from state init -> state preparing >Thread-53::INFO::2014-10-23 17:13:49,305::logUtils::44::dispatcher::(wrapper) Run and protect: getVGInfo(vgUUID=u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', options=None) >Thread-53::INFO::2014-10-23 17:13:49,318::logUtils::47::dispatcher::(wrapper) Run and protect: getVGInfo, Return response: {'info': {'state': 'OK', 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-53::DEBUG::2014-10-23 17:13:49,318::task::1191::Storage.TaskManager.Task::(prepare) Task=`a97f7495-3109-4f2c-a2cc-40446511b0a4`::finished: {'info': {'state': 'OK', 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-53::DEBUG::2014-10-23 17:13:49,318::task::595::Storage.TaskManager.Task::(_updateState) Task=`a97f7495-3109-4f2c-a2cc-40446511b0a4`::moving from state preparing -> state finished >Thread-53::DEBUG::2014-10-23 17:13:49,318::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-53::DEBUG::2014-10-23 17:13:49,318::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-53::DEBUG::2014-10-23 17:13:49,319::task::993::Storage.TaskManager.Task::(_decref) Task=`a97f7495-3109-4f2c-a2cc-40446511b0a4`::ref 0 aborting False >Thread-53::WARNING::2014-10-23 17:13:49,319::Bridge::212::root::(_typeFixup) Fixing up type VolumeGroupInfo >Thread-53::WARNING::2014-10-23 17:13:49,319::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-53::DEBUG::2014-10-23 17:13:49,319::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.getInfo' in bridge with {'state': 'OK', 'deviceType': 3, 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}} >Thread-53::DEBUG::2014-10-23 17:13:49,319::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:49,367::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:49,368::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-54::DEBUG::2014-10-23 17:13:49,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bc63030-1d73-4e90-b9de-9cf2cbd57eba`::moving from state init -> state preparing >Thread-54::INFO::2014-10-23 17:13:49,369::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-54::INFO::2014-10-23 17:13:49,369::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-54::DEBUG::2014-10-23 17:13:49,369::task::1191::Storage.TaskManager.Task::(prepare) Task=`8bc63030-1d73-4e90-b9de-9cf2cbd57eba`::finished: {} >Thread-54::DEBUG::2014-10-23 17:13:49,369::task::595::Storage.TaskManager.Task::(_updateState) Task=`8bc63030-1d73-4e90-b9de-9cf2cbd57eba`::moving from state preparing -> state finished >Thread-54::DEBUG::2014-10-23 17:13:49,369::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-54::DEBUG::2014-10-23 17:13:49,369::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-54::DEBUG::2014-10-23 17:13:49,369::task::993::Storage.TaskManager.Task::(_decref) Task=`8bc63030-1d73-4e90-b9de-9cf2cbd57eba`::ref 0 aborting False >Thread-54::DEBUG::2014-10-23 17:13:49,371::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:49,414::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:49,414::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-55::DEBUG::2014-10-23 17:13:49,415::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:50,373::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:50,374::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-56::DEBUG::2014-10-23 17:13:50,374::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-56::DEBUG::2014-10-23 17:13:50,375::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6c09c3-078d-41ee-8a40-18a47c93cfe7`::moving from state init -> state preparing >Thread-56::INFO::2014-10-23 17:13:50,375::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-56::DEBUG::2014-10-23 17:13:50,375::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-56::DEBUG::2014-10-23 17:13:50,385::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-56::DEBUG::2014-10-23 17:13:50,386::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-56::DEBUG::2014-10-23 17:13:50,397::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-56::DEBUG::2014-10-23 17:13:50,397::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-56::DEBUG::2014-10-23 17:13:50,405::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-56::DEBUG::2014-10-23 17:13:50,405::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f',) >Thread-56::DEBUG::2014-10-23 17:13:50,405::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain} >Thread-56::INFO::2014-10-23 17:13:50,406::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-56::DEBUG::2014-10-23 17:13:50,406::task::1191::Storage.TaskManager.Task::(prepare) Task=`1f6c09c3-078d-41ee-8a40-18a47c93cfe7`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-56::DEBUG::2014-10-23 17:13:50,406::task::595::Storage.TaskManager.Task::(_updateState) Task=`1f6c09c3-078d-41ee-8a40-18a47c93cfe7`::moving from state preparing -> state finished >Thread-56::DEBUG::2014-10-23 17:13:50,406::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-56::DEBUG::2014-10-23 17:13:50,406::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-56::DEBUG::2014-10-23 17:13:50,406::task::993::Storage.TaskManager.Task::(_decref) Task=`1f6c09c3-078d-41ee-8a40-18a47c93cfe7`::ref 0 aborting False >Thread-56::DEBUG::2014-10-23 17:13:50,406::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-56::DEBUG::2014-10-23 17:13:50,406::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:50,418::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:50,418::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-57::DEBUG::2014-10-23 17:13:50,419::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.create' in bridge with {u'masterVersion': 1, u'leaseTimeSec': 60, u'name': u'Default', u'domainList': [u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'], u'lockRenewalIntervalSec': 5, u'leaseRetries': 3, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'ioOpTimeoutSec': 10, u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'} >Thread-57::DEBUG::2014-10-23 17:13:50,420::task::595::Storage.TaskManager.Task::(_updateState) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::moving from state init -> state preparing >Thread-57::INFO::2014-10-23 17:13:50,420::logUtils::44::dispatcher::(wrapper) Run and protect: createStoragePool(poolType=None, spUUID=u'00000002-0002-0002-0002-000000000067', poolName=u'Default', masterDom=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', domList=[u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'], masterVersion=1, lockPolicy=None, lockRenewalIntervalSec=5, leaseTimeSec=60, ioOpTimeoutSec=10, leaseRetries=3, options=None) >Thread-57::DEBUG::2014-10-23 17:13:50,420::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-57::DEBUG::2014-10-23 17:13:50,420::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-57::DEBUG::2014-10-23 17:13:50,420::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-57::DEBUG::2014-10-23 17:13:50,420::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-57::DEBUG::2014-10-23 17:13:50,420::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-57::DEBUG::2014-10-23 17:13:50,420::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,432::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-57::DEBUG::2014-10-23 17:13:50,433::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,514::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,514::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,514::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,515::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,515::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,515::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,515::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,515::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-57::DEBUG::2014-10-23 17:13:50,515::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,516::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,613::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,613::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,614::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-57::DEBUG::2014-10-23 17:13:50,614::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,618::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000490381 s, 4.2 MB/s\n'; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,618::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000490381 s, 4.2 MB/s'], size: 2048 >Thread-57::DEBUG::2014-10-23 17:13:50,618::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-57::DEBUG::2014-10-23 17:13:50,619::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-57::DEBUG::2014-10-23 17:13:50,619::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-57::DEBUG::2014-10-23 17:13:50,619::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,619::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,619::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,619::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,619::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,620::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,712::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,712::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,713::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-57::INFO::2014-10-23 17:13:50,713::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-57::INFO::2014-10-23 17:13:50,713::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-57::INFO::2014-10-23 17:13:50,713::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-57::DEBUG::2014-10-23 17:13:50,714::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`34e3998d-2009-4f2a-98bc-a9dc79f3efc4`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '973' at 'createStoragePool' >Thread-57::DEBUG::2014-10-23 17:13:50,714::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-57::DEBUG::2014-10-23 17:13:50,714::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-57::DEBUG::2014-10-23 17:13:50,714::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`34e3998d-2009-4f2a-98bc-a9dc79f3efc4`::Granted request >Thread-57::DEBUG::2014-10-23 17:13:50,714::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-57::DEBUG::2014-10-23 17:13:50,714::task::993::Storage.TaskManager.Task::(_decref) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::ref 1 aborting False >Thread-57::DEBUG::2014-10-23 17:13:50,715::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`881d5eba-9f14-408c-90c6-9b3fb00d4643`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '975' at 'createStoragePool' >Thread-57::DEBUG::2014-10-23 17:13:50,715::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >Thread-57::DEBUG::2014-10-23 17:13:50,715::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >Thread-57::DEBUG::2014-10-23 17:13:50,715::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`881d5eba-9f14-408c-90c6-9b3fb00d4643`::Granted request >Thread-57::DEBUG::2014-10-23 17:13:50,715::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::_resourcesAcquired: Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (exclusive) >Thread-57::DEBUG::2014-10-23 17:13:50,715::task::993::Storage.TaskManager.Task::(_decref) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::ref 1 aborting False >Thread-57::INFO::2014-10-23 17:13:50,715::sp::133::Storage.StoragePool::(setBackend) updating pool 00000002-0002-0002-0002-000000000067 backend from type NoneType instance 0x3533572210 to type StoragePoolDiskBackend instance 0x7fa0502200e0 >Thread-57::INFO::2014-10-23 17:13:50,716::sp::545::Storage.StoragePool::(create) spUUID=00000002-0002-0002-0002-000000000067 poolName=Default master_sd=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f domList=[u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'] masterVersion=1 {'LEASETIMESEC': 60, 'IOOPTIMEOUTSEC': 10, 'LEASERETRIES': 3, 'LOCKRENEWALINTERVALSEC': 5} >Thread-57::INFO::2014-10-23 17:13:50,716::blockSD::839::Storage.StorageDomain::(validate) sdUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:13:50,716::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,806::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,806::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,806::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,806::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,807::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,807::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,807::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,899::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,899::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,899::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,900::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:13:50,995::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:13:50,996::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:13:50,996::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-57::DEBUG::2014-10-23 17:13:50,996::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-57::DEBUG::2014-10-23 17:13:50,996::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-57::DEBUG::2014-10-23 17:13:50,997::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-57::INFO::2014-10-23 17:13:50,997::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 250) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:52,555::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:52,556::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-58::DEBUG::2014-10-23 17:13:52,557::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:55,570::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:55,570::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-59::DEBUG::2014-10-23 17:13:55,571::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:13:58,600::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:13:58,601::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-60::DEBUG::2014-10-23 17:13:58,602::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:01,627::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:01,628::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-61::DEBUG::2014-10-23 17:14:01,629::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:04,655::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:04,656::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-62::DEBUG::2014-10-23 17:14:04,657::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c90698f-c6e8-48dd-9e31-79f198c977c0`::moving from state init -> state preparing >Thread-62::INFO::2014-10-23 17:14:04,658::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-62::INFO::2014-10-23 17:14:04,658::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-62::DEBUG::2014-10-23 17:14:04,658::task::1191::Storage.TaskManager.Task::(prepare) Task=`6c90698f-c6e8-48dd-9e31-79f198c977c0`::finished: {} >Thread-62::DEBUG::2014-10-23 17:14:04,658::task::595::Storage.TaskManager.Task::(_updateState) Task=`6c90698f-c6e8-48dd-9e31-79f198c977c0`::moving from state preparing -> state finished >Thread-62::DEBUG::2014-10-23 17:14:04,658::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-62::DEBUG::2014-10-23 17:14:04,658::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-62::DEBUG::2014-10-23 17:14:04,658::task::993::Storage.TaskManager.Task::(_decref) Task=`6c90698f-c6e8-48dd-9e31-79f198c977c0`::ref 0 aborting False >Thread-62::DEBUG::2014-10-23 17:14:04,660::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:04,696::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:04,697::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-63::DEBUG::2014-10-23 17:14:04,697::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:07,871::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:07,872::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-64::DEBUG::2014-10-23 17:14:07,873::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:10,889::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:10,890::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-65::DEBUG::2014-10-23 17:14:10,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-57::DEBUG::2014-10-23 17:14:12,000::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 250) >Thread-57::DEBUG::2014-10-23 17:14:12,000::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-57::WARNING::2014-10-23 17:14:12,000::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-57::DEBUG::2014-10-23 17:14:12,000::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-57::WARNING::2014-10-23 17:14:12,000::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-57::DEBUG::2014-10-23 17:14:12,001::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,001::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,001::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,001::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,001::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-57::DEBUG::2014-10-23 17:14:12,001::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,006::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000619244 s, 3.3 MB/s\n'; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,006::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000619244 s, 3.3 MB/s'], size: 2048 >Thread-57::DEBUG::2014-10-23 17:14:12,006::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-57::DEBUG::2014-10-23 17:14:12,006::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-57::DEBUG::2014-10-23 17:14:12,006::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-57::DEBUG::2014-10-23 17:14:12,006::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,007::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,007::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,007::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,007::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,007::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,115::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,116::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,116::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572'] >Thread-57::INFO::2014-10-23 17:14:12,116::clusterlock::295::Storage.SANLock::(acquire) Acquiring cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 250) >Thread-57::DEBUG::2014-10-23 17:14:12,194::clusterlock::321::Storage.SANLock::(acquire) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 250) >Thread-57::INFO::2014-10-23 17:14:12,194::sp::679::Storage.StoragePool::(createMaster) setting master domain for spUUID 00000002-0002-0002-0002-000000000067 on sdUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:14:12,194::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-57::DEBUG::2014-10-23 17:14:12,194::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-57::DEBUG::2014-10-23 17:14:12,194::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', u'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'ROLE=Regular', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', '_SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef'] >Thread-57::DEBUG::2014-10-23 17:14:12,194::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,195::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,195::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,195::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,195::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,195::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,291::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,292::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,292::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_POOL_DESCRIPTION=Default, MDT_POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active, MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef, MDT_POOL_SPM_ID=-1, MDT_POOL_SPM_LVER=-1, MDT_MASTER_VERSION=1 removing=MDT__SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572 >Thread-57::DEBUG::2014-10-23 17:14:12,292::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT__SHA_CKSUM=2bad885ceb5ecbcc9da22e76e5c6a0ac9e889572 --addtag MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef --addtag MDT_POOL_SPM_ID=-1 --addtag MDT_POOL_DESCRIPTION=Default --addtag MDT_POOL_SPM_LVER=-1 --addtag MDT_POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active --addtag MDT_MASTER_VERSION=1 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,480::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,480::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,480::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,480::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-57::DEBUG::2014-10-23 17:14:12,480::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,480::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,481::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,481::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,481::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,481::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,579::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,579::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'ROLE=Regular', '_SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1'] >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-57::DEBUG::2014-10-23 17:14:12,580::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-57::DEBUG::2014-10-23 17:14:12,581::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=00000002-0002-0002-0002-000000000067', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'ROLE=Master', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7'] >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,581::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,675::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,675::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,676::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_ROLE=Master, MDT__SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7, MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 removing=MDT_ROLE=Regular, MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef, MDT_POOL_UUID= >Thread-57::DEBUG::2014-10-23 17:14:12,676::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef --deltag MDT_ROLE=Regular --deltag MDT_POOL_UUID= --addtag MDT_ROLE=Master --addtag MDT__SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7 --addtag MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,847::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,848::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,848::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,848::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-57::DEBUG::2014-10-23 17:14:12,848::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-57::DEBUG::2014-10-23 17:14:12,848::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-57::DEBUG::2014-10-23 17:14:12,848::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,849::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:12,941::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:12,941::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,941::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-57::INFO::2014-10-23 17:14:12,942::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:14:12,942::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` started monitoring domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >Thread-66::DEBUG::2014-10-23 17:14:12,942::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:14:12,943::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-57::DEBUG::2014-10-23 17:14:12,943::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-66::DEBUG::2014-10-23 17:14:12,944::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::WARNING::2014-10-23 17:14:12,944::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-57::DEBUG::2014-10-23 17:14:12,949::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,949::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,950::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,950::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,950::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:12,950::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-66::DEBUG::2014-10-23 17:14:13,036::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-10-23 17:14:13,037::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-66::DEBUG::2014-10-23 17:14:13,041::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000479477 s, 8.5 MB/s\n'; <rc> = 0 >Thread-66::DEBUG::2014-10-23 17:14:13,042::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-66::DEBUG::2014-10-23 17:14:13,042::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-57::DEBUG::2014-10-23 17:14:13,047::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:13,047::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-57::DEBUG::2014-10-23 17:14:13,048::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-57::DEBUG::2014-10-23 17:14:13,048::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-66::DEBUG::2014-10-23 17:14:13,143::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-66::DEBUG::2014-10-23 17:14:13,145::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f changed its status to Valid >Thread-57::DEBUG::2014-10-23 17:14:13,147::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-57::DEBUG::2014-10-23 17:14:13,147::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-57::DEBUG::2014-10-23 17:14:13,147::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-57::WARNING::2014-10-23 17:14:13,147::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-57::DEBUG::2014-10-23 17:14:13,147::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-57::WARNING::2014-10-23 17:14:13,148::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-57::INFO::2014-10-23 17:14:13,148::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:14:13,148::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::INFO::2014-10-23 17:14:13,148::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-57::DEBUG::2014-10-23 17:14:13,148::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-57::INFO::2014-10-23 17:14:13,148::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-57::DEBUG::2014-10-23 17:14:13,200::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully released >Thread-57::INFO::2014-10-23 17:14:13,200::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 250) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:13,922::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:13,924::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-67::DEBUG::2014-10-23 17:14:13,925::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-57::DEBUG::2014-10-23 17:14:15,200::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f released successfully (id: 250) >Thread-57::INFO::2014-10-23 17:14:15,200::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-66::DEBUG::2014-10-23 17:14:15,201::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-66::INFO::2014-10-23 17:14:15,201::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 250) >Thread-66::DEBUG::2014-10-23 17:14:15,201::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f released successfully (id: 250) >Thread-57::INFO::2014-10-23 17:14:15,201::logUtils::47::dispatcher::(wrapper) Run and protect: createStoragePool, Return response: True >Thread-57::DEBUG::2014-10-23 17:14:15,201::task::1191::Storage.TaskManager.Task::(prepare) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::finished: True >Thread-57::DEBUG::2014-10-23 17:14:15,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::moving from state preparing -> state finished >Thread-57::DEBUG::2014-10-23 17:14:15,201::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': < ResourceRef 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-57::DEBUG::2014-10-23 17:14:15,202::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-57::DEBUG::2014-10-23 17:14:15,203::task::993::Storage.TaskManager.Task::(_decref) Task=`26258b98-e96d-472e-9080-7a9264b82f9a`::ref 0 aborting False >Thread-57::DEBUG::2014-10-23 17:14:15,203::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.create' in bridge with True >Thread-57::DEBUG::2014-10-23 17:14:15,203::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:15,283::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-68::DEBUG::2014-10-23 17:14:15,284::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >JsonRpcServer::DEBUG::2014-10-23 17:14:15,284::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-68::DEBUG::2014-10-23 17:14:15,285::task::595::Storage.TaskManager.Task::(_updateState) Task=`99438dda-3d7e-4513-a38b-1fba894cd30f`::moving from state init -> state preparing >Thread-68::INFO::2014-10-23 17:14:15,285::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-68::DEBUG::2014-10-23 17:14:15,286::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-68::DEBUG::2014-10-23 17:14:15,295::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-68::DEBUG::2014-10-23 17:14:15,296::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-68::DEBUG::2014-10-23 17:14:15,304::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-68::DEBUG::2014-10-23 17:14:15,304::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-68::DEBUG::2014-10-23 17:14:15,312::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-68::DEBUG::2014-10-23 17:14:15,313::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f',) >Thread-68::DEBUG::2014-10-23 17:14:15,313::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain} >Thread-68::INFO::2014-10-23 17:14:15,313::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-68::DEBUG::2014-10-23 17:14:15,313::task::1191::Storage.TaskManager.Task::(prepare) Task=`99438dda-3d7e-4513-a38b-1fba894cd30f`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-68::DEBUG::2014-10-23 17:14:15,313::task::595::Storage.TaskManager.Task::(_updateState) Task=`99438dda-3d7e-4513-a38b-1fba894cd30f`::moving from state preparing -> state finished >Thread-68::DEBUG::2014-10-23 17:14:15,313::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-68::DEBUG::2014-10-23 17:14:15,313::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-68::DEBUG::2014-10-23 17:14:15,313::task::993::Storage.TaskManager.Task::(_decref) Task=`99438dda-3d7e-4513-a38b-1fba894cd30f`::ref 0 aborting False >Thread-68::DEBUG::2014-10-23 17:14:15,314::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-68::DEBUG::2014-10-23 17:14:15,314::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:15,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:15,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-69::DEBUG::2014-10-23 17:14:15,351::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.getInfo' in bridge with {u'lvmvolumegroupID': u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP'} >Thread-69::DEBUG::2014-10-23 17:14:15,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`676e4b7b-a5b0-4f33-be97-12f8d6d42b22`::moving from state init -> state preparing >Thread-69::INFO::2014-10-23 17:14:15,352::logUtils::44::dispatcher::(wrapper) Run and protect: getVGInfo(vgUUID=u'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', options=None) >Thread-69::DEBUG::2014-10-23 17:14:15,376::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-69::DEBUG::2014-10-23 17:14:15,376::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size /dev/mapper/1cmg01 (cwd None) >Thread-69::DEBUG::2014-10-23 17:14:15,475::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-69::DEBUG::2014-10-23 17:14:15,476::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-69::INFO::2014-10-23 17:14:15,476::logUtils::47::dispatcher::(wrapper) Run and protect: getVGInfo, Return response: {'info': {'state': 'OK', 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-69::DEBUG::2014-10-23 17:14:15,476::task::1191::Storage.TaskManager.Task::(prepare) Task=`676e4b7b-a5b0-4f33-be97-12f8d6d42b22`::finished: {'info': {'state': 'OK', 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-69::DEBUG::2014-10-23 17:14:15,476::task::595::Storage.TaskManager.Task::(_updateState) Task=`676e4b7b-a5b0-4f33-be97-12f8d6d42b22`::moving from state preparing -> state finished >Thread-69::DEBUG::2014-10-23 17:14:15,476::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-69::DEBUG::2014-10-23 17:14:15,476::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-69::DEBUG::2014-10-23 17:14:15,477::task::993::Storage.TaskManager.Task::(_decref) Task=`676e4b7b-a5b0-4f33-be97-12f8d6d42b22`::ref 0 aborting False >Thread-69::WARNING::2014-10-23 17:14:15,477::Bridge::212::root::(_typeFixup) Fixing up type VolumeGroupInfo >Thread-69::WARNING::2014-10-23 17:14:15,477::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-69::DEBUG::2014-10-23 17:14:15,477::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.getInfo' in bridge with {'state': 'OK', 'deviceType': 3, 'vgsize': '90865401856', 'name': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'vgfree': '86704652288', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '90865401856', 'fwrev': '0000', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'devcapacity': '91268055040', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}} >Thread-69::DEBUG::2014-10-23 17:14:15,478::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:15,599::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:15,600::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-70::DEBUG::2014-10-23 17:14:15,600::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-70::DEBUG::2014-10-23 17:14:15,601::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::moving from state init -> state preparing >Thread-70::INFO::2014-10-23 17:14:15,601::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-70::ERROR::2014-10-23 17:14:15,601::task::866::Storage.TaskManager.Task::(_setError) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::Unexpected error >Traceback (most recent call last): > File "/usr/share/vdsm/storage/task.py", line 873, in _run > return fn(*args, **kargs) > File "/usr/share/vdsm/logUtils.py", line 45, in wrapper > res = f(*args, **kwargs) > File "/usr/share/vdsm/storage/hsm.py", line 609, in getSpmStatus > pool = self.getPool(spUUID) > File "/usr/share/vdsm/storage/hsm.py", line 325, in getPool > raise se.StoragePoolUnknown(spUUID) >StoragePoolUnknown: Unknown pool id, pool not connected: (u'00000002-0002-0002-0002-000000000067',) >Thread-70::DEBUG::2014-10-23 17:14:15,602::task::885::Storage.TaskManager.Task::(_run) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::Task._run: ec5decdd-7944-48ff-8e0d-1bcf27b4d51d (u'00000002-0002-0002-0002-000000000067',) {} failed - stopping task >Thread-70::DEBUG::2014-10-23 17:14:15,602::task::1217::Storage.TaskManager.Task::(stop) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::stopping in state preparing (force False) >Thread-70::DEBUG::2014-10-23 17:14:15,602::task::993::Storage.TaskManager.Task::(_decref) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::ref 1 aborting True >Thread-70::INFO::2014-10-23 17:14:15,602::task::1171::Storage.TaskManager.Task::(prepare) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::aborting: Task is aborted: 'Unknown pool id, pool not connected' - code 309 >Thread-70::DEBUG::2014-10-23 17:14:15,602::task::1176::Storage.TaskManager.Task::(prepare) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::Prepare: aborted: Unknown pool id, pool not connected >Thread-70::DEBUG::2014-10-23 17:14:15,603::task::993::Storage.TaskManager.Task::(_decref) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::ref 0 aborting True >Thread-70::DEBUG::2014-10-23 17:14:15,603::task::928::Storage.TaskManager.Task::(_doAbort) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::Task._doAbort: force False >Thread-70::DEBUG::2014-10-23 17:14:15,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-70::DEBUG::2014-10-23 17:14:15,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::moving from state preparing -> state aborting >Thread-70::DEBUG::2014-10-23 17:14:15,603::task::550::Storage.TaskManager.Task::(__state_aborting) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::_aborting: recover policy none >Thread-70::DEBUG::2014-10-23 17:14:15,603::task::595::Storage.TaskManager.Task::(_updateState) Task=`ec5decdd-7944-48ff-8e0d-1bcf27b4d51d`::moving from state aborting -> state failed >Thread-70::DEBUG::2014-10-23 17:14:15,603::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-70::DEBUG::2014-10-23 17:14:15,603::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-70::ERROR::2014-10-23 17:14:15,603::dispatcher::76::Storage.Dispatcher::(wrapper) {'status': {'message': "Unknown pool id, pool not connected: (u'00000002-0002-0002-0002-000000000067',)", 'code': 309}} >Thread-70::DEBUG::2014-10-23 17:14:15,603::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:15,634::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:15,635::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-71::DEBUG::2014-10-23 17:14:15,635::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-71::DEBUG::2014-10-23 17:14:15,636::task::595::Storage.TaskManager.Task::(_updateState) Task=`b48c7289-a0e2-497a-b91c-703eed0daaa7`::moving from state init -> state preparing >Thread-71::INFO::2014-10-23 17:14:15,636::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, options=None) >Thread-71::DEBUG::2014-10-23 17:14:15,636::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`2ffea9dc-3063-4915-8ea2-8f4edd0e3673`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-71::DEBUG::2014-10-23 17:14:15,636::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-71::DEBUG::2014-10-23 17:14:15,636::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-71::DEBUG::2014-10-23 17:14:15,636::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`2ffea9dc-3063-4915-8ea2-8f4edd0e3673`::Granted request >Thread-71::DEBUG::2014-10-23 17:14:15,637::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`89a9dc34-c968-4fbc-847e-06cf7cd657d9`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-71::DEBUG::2014-10-23 17:14:15,637::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-71::DEBUG::2014-10-23 17:14:15,637::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-71::DEBUG::2014-10-23 17:14:15,637::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`89a9dc34-c968-4fbc-847e-06cf7cd657d9`::Granted request >Thread-71::DEBUG::2014-10-23 17:14:15,637::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'} >Thread-71::INFO::2014-10-23 17:14:15,637::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active'} >Thread-71::INFO::2014-10-23 17:14:15,637::sp::133::Storage.StoragePool::(setBackend) updating pool 00000002-0002-0002-0002-000000000067 backend from type NoneType instance 0x3533572210 to type StoragePoolMemoryBackend instance 0x7fa05c462af8 >Thread-71::INFO::2014-10-23 17:14:15,637::sp::631::Storage.StoragePool::(connect) Connect host #1 to the storage pool 00000002-0002-0002-0002-000000000067 with master domain: 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (ver = 1) >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,638::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-71::DEBUG::2014-10-23 17:14:15,638::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-71::DEBUG::2014-10-23 17:14:15,638::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-71::DEBUG::2014-10-23 17:14:15,638::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-71::DEBUG::2014-10-23 17:14:15,639::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-71::DEBUG::2014-10-23 17:14:15,639::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,649::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-71::DEBUG::2014-10-23 17:14:15,650::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,732::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:15,732::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,732::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,732::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,733::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,733::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,733::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,733::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-71::DEBUG::2014-10-23 17:14:15,733::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,734::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,825::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:15,826::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,826::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-71::DEBUG::2014-10-23 17:14:15,826::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,830::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000381266 s, 5.4 MB/s\n'; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:15,830::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000381266 s, 5.4 MB/s'], size: 2048 >Thread-71::DEBUG::2014-10-23 17:14:15,830::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-71::DEBUG::2014-10-23 17:14:15,830::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-71::DEBUG::2014-10-23 17:14:15,831::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,831::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,923::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:15,924::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,924::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-71::INFO::2014-10-23 17:14:15,924::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-71::INFO::2014-10-23 17:14:15,924::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-71::INFO::2014-10-23 17:14:15,924::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-71::DEBUG::2014-10-23 17:14:15,925::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-71::DEBUG::2014-10-23 17:14:15,925::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-71::DEBUG::2014-10-23 17:14:15,925::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-71::INFO::2014-10-23 17:14:15,925::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-71::DEBUG::2014-10-23 17:14:15,925::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` started monitoring domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >Thread-72::DEBUG::2014-10-23 17:14:15,925::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-71::DEBUG::2014-10-23 17:14:15,926::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-71::DEBUG::2014-10-23 17:14:15,926::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-71::WARNING::2014-10-23 17:14:15,927::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-72::DEBUG::2014-10-23 17:14:15,927::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:15,931::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:15,932::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:16,020::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:16,020::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:16,024::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00039765 s, 10.3 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:16,026::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f changed its status to Valid >Thread-73::DEBUG::2014-10-23 17:14:16,026::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-72::INFO::2014-10-23 17:14:16,026::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >Thread-73::DEBUG::2014-10-23 17:14:16,027::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-72::DEBUG::2014-10-23 17:14:16,028::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >Thread-73::DEBUG::2014-10-23 17:14:16,028::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-71::DEBUG::2014-10-23 17:14:16,029::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:16,030::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-71::DEBUG::2014-10-23 17:14:16,030::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-71::WARNING::2014-10-23 17:14:16,030::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-71::DEBUG::2014-10-23 17:14:16,030::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-71::WARNING::2014-10-23 17:14:16,030::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-71::INFO::2014-10-23 17:14:16,031::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-71::DEBUG::2014-10-23 17:14:16,031::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-71::INFO::2014-10-23 17:14:16,031::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-71::DEBUG::2014-10-23 17:14:16,031::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-71::DEBUG::2014-10-23 17:14:16,031::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-71::DEBUG::2014-10-23 17:14:16,033::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=8 (cwd None) >Thread-74::INFO::2014-10-23 17:14:16,036::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-71::DEBUG::2014-10-23 17:14:16,039::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00219521 s, 1.9 MB/s\n'; <rc> = 0 >Thread-71::INFO::2014-10-23 17:14:16,039::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-71::DEBUG::2014-10-23 17:14:16,039::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-71::DEBUG::2014-10-23 17:14:16,045::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00243333 s, 1.7 MB/s\n'; <rc> = 0 >Thread-71::DEBUG::2014-10-23 17:14:16,047::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 00000002-0002-0002-0002-000000000067 >Thread-71::DEBUG::2014-10-23 17:14:16,048::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 00000002-0002-0002-0002-000000000067 on master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-71::DEBUG::2014-10-23 17:14:16,049::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-71::DEBUG::2014-10-23 17:14:16,050::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-71::DEBUG::2014-10-23 17:14:16,050::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-71::DEBUG::2014-10-23 17:14:16,050::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-71::DEBUG::2014-10-23 17:14:16,050::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-71::DEBUG::2014-10-23 17:14:16,051::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-71::DEBUG::2014-10-23 17:14:16,051::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-71::DEBUG::2014-10-23 17:14:16,051::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-71::INFO::2014-10-23 17:14:16,051::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-71::DEBUG::2014-10-23 17:14:16,051::task::1191::Storage.TaskManager.Task::(prepare) Task=`b48c7289-a0e2-497a-b91c-703eed0daaa7`::finished: True >Thread-71::DEBUG::2014-10-23 17:14:16,051::task::595::Storage.TaskManager.Task::(_updateState) Task=`b48c7289-a0e2-497a-b91c-703eed0daaa7`::moving from state preparing -> state finished >Thread-71::DEBUG::2014-10-23 17:14:16,051::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-71::DEBUG::2014-10-23 17:14:16,051::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-71::DEBUG::2014-10-23 17:14:16,051::task::993::Storage.TaskManager.Task::(_decref) Task=`b48c7289-a0e2-497a-b91c-703eed0daaa7`::ref 0 aborting False >Thread-71::DEBUG::2014-10-23 17:14:16,052::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-71::DEBUG::2014-10-23 17:14:16,052::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:16,061::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-81::DEBUG::2014-10-23 17:14:16,062::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:14:16,062::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-81::DEBUG::2014-10-23 17:14:16,064::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f2e93ef-b06a-47d9-b12b-cd6aab9032dc`::moving from state init -> state preparing >Thread-81::INFO::2014-10-23 17:14:16,064::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-81::INFO::2014-10-23 17:14:16,075::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-81::DEBUG::2014-10-23 17:14:16,075::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f2e93ef-b06a-47d9-b12b-cd6aab9032dc`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-81::DEBUG::2014-10-23 17:14:16,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f2e93ef-b06a-47d9-b12b-cd6aab9032dc`::moving from state preparing -> state finished >Thread-81::DEBUG::2014-10-23 17:14:16,075::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-81::DEBUG::2014-10-23 17:14:16,075::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-81::DEBUG::2014-10-23 17:14:16,075::task::993::Storage.TaskManager.Task::(_decref) Task=`3f2e93ef-b06a-47d9-b12b-cd6aab9032dc`::ref 0 aborting False >Thread-81::DEBUG::2014-10-23 17:14:16,075::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1} >Thread-81::DEBUG::2014-10-23 17:14:16,075::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:16,198::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:16,199::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-82::DEBUG::2014-10-23 17:14:16,199::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.spmStart' in bridge with {u'maxHostID': 250, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'prevLver': u'-1', u'enableScsiFencing': u'false', u'prevID': -1} >Thread-82::DEBUG::2014-10-23 17:14:16,200::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::moving from state init -> state preparing >Thread-82::INFO::2014-10-23 17:14:16,200::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID=u'00000002-0002-0002-0002-000000000067', prevID=-1, prevLVER=u'-1', maxHostID=250, domVersion=None, options=None) >Thread-82::DEBUG::2014-10-23 17:14:16,200::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`0378f594-04b8-4838-998f-33c820399bb2`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '572' at 'spmStart' >Thread-82::DEBUG::2014-10-23 17:14:16,200::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-82::DEBUG::2014-10-23 17:14:16,200::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-82::DEBUG::2014-10-23 17:14:16,200::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`0378f594-04b8-4838-998f-33c820399bb2`::Granted request >Thread-82::DEBUG::2014-10-23 17:14:16,201::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-82::DEBUG::2014-10-23 17:14:16,201::task::993::Storage.TaskManager.Task::(_decref) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::ref 1 aborting False >Thread-82::DEBUG::2014-10-23 17:14:16,201::taskManager::78::Storage.TaskManager::(scheduleJob) scheduled job spmStart for task 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-82::INFO::2014-10-23 17:14:16,201::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-82::DEBUG::2014-10-23 17:14:16,201::task::1183::Storage.TaskManager.Task::(prepare) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::Prepare: 1 jobs exist, move to acquiring >Thread-82::DEBUG::2014-10-23 17:14:16,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::moving from state preparing -> state acquiring >Thread-82::DEBUG::2014-10-23 17:14:16,201::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::moving from state acquiring -> state queued >Thread-82::DEBUG::2014-10-23 17:14:16,201::taskManager::56::Storage.TaskManager::(_queueTask) queuing task: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-82::DEBUG::2014-10-23 17:14:16,201::taskManager::64::Storage.TaskManager::(_queueTask) task queued: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-8::DEBUG::2014-10-23 17:14:16,202::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-82::DEBUG::2014-10-23 17:14:16,202::task::1188::Storage.TaskManager.Task::(prepare) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::returning >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,202::threadPool::200::Storage.ThreadPool.WorkerThread::(_processNextTask) Task: 1c557727-9394-4848-9699-e6a94a5708d6 running: <bound method Task.commit of <storage.task.Task instance at 0x7fa05c42e9e0>> with: None >Thread-82::DEBUG::2014-10-23 17:14:16,202::task::993::Storage.TaskManager.Task::(_decref) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::ref 0 aborting False >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,202::task::1200::Storage.TaskManager.Task::(commit) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::committing task: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-82::DEBUG::2014-10-23 17:14:16,203::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.spmStart' in bridge with 1c557727-9394-4848-9699-e6a94a5708d6 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,203::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::moving from state queued -> state running >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,203::task::905::Storage.TaskManager.Task::(_runJobs) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fa0500a5190>> (args: (-1, u'-1', 250, None) kwargs: {}) >Thread-82::DEBUG::2014-10-23 17:14:16,204::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,204::task::331::Storage.TaskManager.Task::(run) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fa0500a5190>> (args: (-1, u'-1', 250, None) kwargs: {}) callback None >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,204::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,205::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:16,205::misc::751::Storage.SamplingMethod::(__call__) Returning last result >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:16,216::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:16,951::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:16,952::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-83::DEBUG::2014-10-23 17:14:16,953::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:17,212::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:17,213::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-84::DEBUG::2014-10-23 17:14:17,213::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-84::DEBUG::2014-10-23 17:14:17,213::task::595::Storage.TaskManager.Task::(_updateState) Task=`7185d15e-bc94-40fb-9605-6fd532b27555`::moving from state init -> state preparing >Thread-84::INFO::2014-10-23 17:14:17,213::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-84::DEBUG::2014-10-23 17:14:17,214::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-84::DEBUG::2014-10-23 17:14:17,214::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-84::INFO::2014-10-23 17:14:17,214::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-84::DEBUG::2014-10-23 17:14:17,214::task::1191::Storage.TaskManager.Task::(prepare) Task=`7185d15e-bc94-40fb-9605-6fd532b27555`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-84::DEBUG::2014-10-23 17:14:17,214::task::595::Storage.TaskManager.Task::(_updateState) Task=`7185d15e-bc94-40fb-9605-6fd532b27555`::moving from state preparing -> state finished >Thread-84::DEBUG::2014-10-23 17:14:17,214::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-84::DEBUG::2014-10-23 17:14:17,214::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-84::DEBUG::2014-10-23 17:14:17,214::task::993::Storage.TaskManager.Task::(_decref) Task=`7185d15e-bc94-40fb-9605-6fd532b27555`::ref 0 aborting False >Thread-84::DEBUG::2014-10-23 17:14:17,214::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-84::DEBUG::2014-10-23 17:14:17,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:18,219::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:18,220::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-85::DEBUG::2014-10-23 17:14:18,220::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-85::DEBUG::2014-10-23 17:14:18,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4c0d3e3-2da4-4b71-b34a-fbc1e2a57985`::moving from state init -> state preparing >Thread-85::INFO::2014-10-23 17:14:18,221::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-85::DEBUG::2014-10-23 17:14:18,221::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-85::DEBUG::2014-10-23 17:14:18,221::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-85::INFO::2014-10-23 17:14:18,221::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-85::DEBUG::2014-10-23 17:14:18,221::task::1191::Storage.TaskManager.Task::(prepare) Task=`c4c0d3e3-2da4-4b71-b34a-fbc1e2a57985`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-85::DEBUG::2014-10-23 17:14:18,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`c4c0d3e3-2da4-4b71-b34a-fbc1e2a57985`::moving from state preparing -> state finished >Thread-85::DEBUG::2014-10-23 17:14:18,221::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-85::DEBUG::2014-10-23 17:14:18,221::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-85::DEBUG::2014-10-23 17:14:18,222::task::993::Storage.TaskManager.Task::(_decref) Task=`c4c0d3e3-2da4-4b71-b34a-fbc1e2a57985`::ref 0 aborting False >Thread-85::DEBUG::2014-10-23 17:14:18,222::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-85::DEBUG::2014-10-23 17:14:18,222::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:19,230::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:19,230::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-86::DEBUG::2014-10-23 17:14:19,231::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-86::DEBUG::2014-10-23 17:14:19,231::task::595::Storage.TaskManager.Task::(_updateState) Task=`27c50888-8a4e-4804-ad46-656f9226c2a4`::moving from state init -> state preparing >Thread-86::INFO::2014-10-23 17:14:19,232::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-86::DEBUG::2014-10-23 17:14:19,232::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-86::DEBUG::2014-10-23 17:14:19,232::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-86::INFO::2014-10-23 17:14:19,232::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-86::DEBUG::2014-10-23 17:14:19,232::task::1191::Storage.TaskManager.Task::(prepare) Task=`27c50888-8a4e-4804-ad46-656f9226c2a4`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-86::DEBUG::2014-10-23 17:14:19,232::task::595::Storage.TaskManager.Task::(_updateState) Task=`27c50888-8a4e-4804-ad46-656f9226c2a4`::moving from state preparing -> state finished >Thread-86::DEBUG::2014-10-23 17:14:19,232::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-86::DEBUG::2014-10-23 17:14:19,232::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-86::DEBUG::2014-10-23 17:14:19,232::task::993::Storage.TaskManager.Task::(_decref) Task=`27c50888-8a4e-4804-ad46-656f9226c2a4`::ref 0 aborting False >Thread-86::DEBUG::2014-10-23 17:14:19,232::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-86::DEBUG::2014-10-23 17:14:19,233::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:19,989::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:19,989::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-87::DEBUG::2014-10-23 17:14:19,990::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e0b4f42-d1eb-4d67-9851-909f7f18c427`::moving from state init -> state preparing >Thread-87::INFO::2014-10-23 17:14:19,990::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-87::INFO::2014-10-23 17:14:19,991::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00039765', 'lastCheck': '4.0', 'valid': True}} >Thread-87::DEBUG::2014-10-23 17:14:19,991::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e0b4f42-d1eb-4d67-9851-909f7f18c427`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00039765', 'lastCheck': '4.0', 'valid': True}} >Thread-87::DEBUG::2014-10-23 17:14:19,991::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e0b4f42-d1eb-4d67-9851-909f7f18c427`::moving from state preparing -> state finished >Thread-87::DEBUG::2014-10-23 17:14:19,991::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-87::DEBUG::2014-10-23 17:14:19,991::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-87::DEBUG::2014-10-23 17:14:19,991::task::993::Storage.TaskManager.Task::(_decref) Task=`6e0b4f42-d1eb-4d67-9851-909f7f18c427`::ref 0 aborting False >Thread-87::DEBUG::2014-10-23 17:14:19,993::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:20,026::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:20,027::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-88::DEBUG::2014-10-23 17:14:20,028::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:20,235::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:20,236::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-89::DEBUG::2014-10-23 17:14:20,236::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-89::DEBUG::2014-10-23 17:14:20,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffb608b5-1a4a-4fba-9dbd-f2e3ca05fb7f`::moving from state init -> state preparing >Thread-89::INFO::2014-10-23 17:14:20,236::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-89::DEBUG::2014-10-23 17:14:20,237::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-89::DEBUG::2014-10-23 17:14:20,237::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-89::INFO::2014-10-23 17:14:20,237::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-89::DEBUG::2014-10-23 17:14:20,237::task::1191::Storage.TaskManager.Task::(prepare) Task=`ffb608b5-1a4a-4fba-9dbd-f2e3ca05fb7f`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-89::DEBUG::2014-10-23 17:14:20,237::task::595::Storage.TaskManager.Task::(_updateState) Task=`ffb608b5-1a4a-4fba-9dbd-f2e3ca05fb7f`::moving from state preparing -> state finished >Thread-89::DEBUG::2014-10-23 17:14:20,237::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-89::DEBUG::2014-10-23 17:14:20,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-89::DEBUG::2014-10-23 17:14:20,237::task::993::Storage.TaskManager.Task::(_decref) Task=`ffb608b5-1a4a-4fba-9dbd-f2e3ca05fb7f`::ref 0 aborting False >Thread-89::DEBUG::2014-10-23 17:14:20,237::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-89::DEBUG::2014-10-23 17:14:20,237::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:21,240::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:21,240::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-90::DEBUG::2014-10-23 17:14:21,240::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-90::DEBUG::2014-10-23 17:14:21,241::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e3e2371-3d02-44ac-8156-e92befbc96db`::moving from state init -> state preparing >Thread-90::INFO::2014-10-23 17:14:21,241::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-90::DEBUG::2014-10-23 17:14:21,242::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-90::DEBUG::2014-10-23 17:14:21,242::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-90::INFO::2014-10-23 17:14:21,242::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-90::DEBUG::2014-10-23 17:14:21,242::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e3e2371-3d02-44ac-8156-e92befbc96db`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-90::DEBUG::2014-10-23 17:14:21,242::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e3e2371-3d02-44ac-8156-e92befbc96db`::moving from state preparing -> state finished >Thread-90::DEBUG::2014-10-23 17:14:21,242::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-90::DEBUG::2014-10-23 17:14:21,242::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-90::DEBUG::2014-10-23 17:14:21,242::task::993::Storage.TaskManager.Task::(_decref) Task=`6e3e2371-3d02-44ac-8156-e92befbc96db`::ref 0 aborting False >Thread-90::DEBUG::2014-10-23 17:14:21,242::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-90::DEBUG::2014-10-23 17:14:21,242::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:22,247::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:22,248::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-91::DEBUG::2014-10-23 17:14:22,248::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-91::DEBUG::2014-10-23 17:14:22,249::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b0b4007-bbc2-45e4-b5df-fed8e88cf88c`::moving from state init -> state preparing >Thread-91::INFO::2014-10-23 17:14:22,249::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-91::DEBUG::2014-10-23 17:14:22,249::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-91::DEBUG::2014-10-23 17:14:22,249::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-91::INFO::2014-10-23 17:14:22,249::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-91::DEBUG::2014-10-23 17:14:22,249::task::1191::Storage.TaskManager.Task::(prepare) Task=`2b0b4007-bbc2-45e4-b5df-fed8e88cf88c`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-91::DEBUG::2014-10-23 17:14:22,249::task::595::Storage.TaskManager.Task::(_updateState) Task=`2b0b4007-bbc2-45e4-b5df-fed8e88cf88c`::moving from state preparing -> state finished >Thread-91::DEBUG::2014-10-23 17:14:22,249::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-91::DEBUG::2014-10-23 17:14:22,250::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-91::DEBUG::2014-10-23 17:14:22,250::task::993::Storage.TaskManager.Task::(_decref) Task=`2b0b4007-bbc2-45e4-b5df-fed8e88cf88c`::ref 0 aborting False >Thread-91::DEBUG::2014-10-23 17:14:22,250::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-91::DEBUG::2014-10-23 17:14:22,250::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:23,256::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:23,257::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-92::DEBUG::2014-10-23 17:14:23,257::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-92::DEBUG::2014-10-23 17:14:23,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e5de53f-7801-4347-9707-aa920437335f`::moving from state init -> state preparing >Thread-92::INFO::2014-10-23 17:14:23,259::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-92::DEBUG::2014-10-23 17:14:23,259::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-92::DEBUG::2014-10-23 17:14:23,259::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-92::INFO::2014-10-23 17:14:23,259::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-92::DEBUG::2014-10-23 17:14:23,259::task::1191::Storage.TaskManager.Task::(prepare) Task=`4e5de53f-7801-4347-9707-aa920437335f`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-92::DEBUG::2014-10-23 17:14:23,259::task::595::Storage.TaskManager.Task::(_updateState) Task=`4e5de53f-7801-4347-9707-aa920437335f`::moving from state preparing -> state finished >Thread-92::DEBUG::2014-10-23 17:14:23,260::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-92::DEBUG::2014-10-23 17:14:23,260::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-92::DEBUG::2014-10-23 17:14:23,260::task::993::Storage.TaskManager.Task::(_decref) Task=`4e5de53f-7801-4347-9707-aa920437335f`::ref 0 aborting False >Thread-92::DEBUG::2014-10-23 17:14:23,260::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-92::DEBUG::2014-10-23 17:14:23,260::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:23,266::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:23,266::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-93::DEBUG::2014-10-23 17:14:23,267::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:24,263::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:24,263::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-94::DEBUG::2014-10-23 17:14:24,264::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-94::DEBUG::2014-10-23 17:14:24,264::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eb9b4b3-29b0-4271-a22d-b59497c3fe13`::moving from state init -> state preparing >Thread-94::INFO::2014-10-23 17:14:24,264::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-94::DEBUG::2014-10-23 17:14:24,264::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-94::DEBUG::2014-10-23 17:14:24,265::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-94::INFO::2014-10-23 17:14:24,265::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-94::DEBUG::2014-10-23 17:14:24,265::task::1191::Storage.TaskManager.Task::(prepare) Task=`0eb9b4b3-29b0-4271-a22d-b59497c3fe13`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-94::DEBUG::2014-10-23 17:14:24,265::task::595::Storage.TaskManager.Task::(_updateState) Task=`0eb9b4b3-29b0-4271-a22d-b59497c3fe13`::moving from state preparing -> state finished >Thread-94::DEBUG::2014-10-23 17:14:24,265::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-94::DEBUG::2014-10-23 17:14:24,265::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-94::DEBUG::2014-10-23 17:14:24,265::task::993::Storage.TaskManager.Task::(_decref) Task=`0eb9b4b3-29b0-4271-a22d-b59497c3fe13`::ref 0 aborting False >Thread-94::DEBUG::2014-10-23 17:14:24,265::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-94::DEBUG::2014-10-23 17:14:24,265::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:25,268::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:25,269::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-95::DEBUG::2014-10-23 17:14:25,269::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-95::DEBUG::2014-10-23 17:14:25,269::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e644b0f-37b1-4fac-83a3-55bae7b01eda`::moving from state init -> state preparing >Thread-95::INFO::2014-10-23 17:14:25,269::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-95::DEBUG::2014-10-23 17:14:25,270::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-95::DEBUG::2014-10-23 17:14:25,270::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-95::INFO::2014-10-23 17:14:25,270::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-95::DEBUG::2014-10-23 17:14:25,270::task::1191::Storage.TaskManager.Task::(prepare) Task=`6e644b0f-37b1-4fac-83a3-55bae7b01eda`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-95::DEBUG::2014-10-23 17:14:25,270::task::595::Storage.TaskManager.Task::(_updateState) Task=`6e644b0f-37b1-4fac-83a3-55bae7b01eda`::moving from state preparing -> state finished >Thread-95::DEBUG::2014-10-23 17:14:25,270::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-95::DEBUG::2014-10-23 17:14:25,270::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-95::DEBUG::2014-10-23 17:14:25,270::task::993::Storage.TaskManager.Task::(_decref) Task=`6e644b0f-37b1-4fac-83a3-55bae7b01eda`::ref 0 aborting False >Thread-95::DEBUG::2014-10-23 17:14:25,270::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-95::DEBUG::2014-10-23 17:14:25,270::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:14:26,029::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:26,034::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000560598 s, 7.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:26,273::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:26,274::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-96::DEBUG::2014-10-23 17:14:26,274::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-96::DEBUG::2014-10-23 17:14:26,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`a88b9698-c917-43dc-b05a-fda6888b8ad1`::moving from state init -> state preparing >Thread-96::INFO::2014-10-23 17:14:26,276::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-96::DEBUG::2014-10-23 17:14:26,276::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-96::DEBUG::2014-10-23 17:14:26,276::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-96::INFO::2014-10-23 17:14:26,276::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-96::DEBUG::2014-10-23 17:14:26,276::task::1191::Storage.TaskManager.Task::(prepare) Task=`a88b9698-c917-43dc-b05a-fda6888b8ad1`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-96::DEBUG::2014-10-23 17:14:26,276::task::595::Storage.TaskManager.Task::(_updateState) Task=`a88b9698-c917-43dc-b05a-fda6888b8ad1`::moving from state preparing -> state finished >Thread-96::DEBUG::2014-10-23 17:14:26,277::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-96::DEBUG::2014-10-23 17:14:26,277::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-96::DEBUG::2014-10-23 17:14:26,277::task::993::Storage.TaskManager.Task::(_decref) Task=`a88b9698-c917-43dc-b05a-fda6888b8ad1`::ref 0 aborting False >Thread-96::DEBUG::2014-10-23 17:14:26,277::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-96::DEBUG::2014-10-23 17:14:26,277::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:26,293::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:26,294::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-97::DEBUG::2014-10-23 17:14:26,294::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:27,280::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:27,280::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-98::DEBUG::2014-10-23 17:14:27,280::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-98::DEBUG::2014-10-23 17:14:27,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`67d33d3d-1235-42f8-b9f5-9674b7c31c66`::moving from state init -> state preparing >Thread-98::INFO::2014-10-23 17:14:27,281::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-98::DEBUG::2014-10-23 17:14:27,281::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-98::DEBUG::2014-10-23 17:14:27,281::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-98::INFO::2014-10-23 17:14:27,281::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-98::DEBUG::2014-10-23 17:14:27,281::task::1191::Storage.TaskManager.Task::(prepare) Task=`67d33d3d-1235-42f8-b9f5-9674b7c31c66`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-98::DEBUG::2014-10-23 17:14:27,282::task::595::Storage.TaskManager.Task::(_updateState) Task=`67d33d3d-1235-42f8-b9f5-9674b7c31c66`::moving from state preparing -> state finished >Thread-98::DEBUG::2014-10-23 17:14:27,282::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-98::DEBUG::2014-10-23 17:14:27,282::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-98::DEBUG::2014-10-23 17:14:27,282::task::993::Storage.TaskManager.Task::(_decref) Task=`67d33d3d-1235-42f8-b9f5-9674b7c31c66`::ref 0 aborting False >Thread-98::DEBUG::2014-10-23 17:14:27,282::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-98::DEBUG::2014-10-23 17:14:27,282::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:28,285::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:28,286::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-99::DEBUG::2014-10-23 17:14:28,286::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-99::DEBUG::2014-10-23 17:14:28,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9b66af-9a1b-49c9-ad31-193c8daac3ae`::moving from state init -> state preparing >Thread-99::INFO::2014-10-23 17:14:28,287::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-99::DEBUG::2014-10-23 17:14:28,287::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-99::DEBUG::2014-10-23 17:14:28,287::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-99::INFO::2014-10-23 17:14:28,287::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-99::DEBUG::2014-10-23 17:14:28,287::task::1191::Storage.TaskManager.Task::(prepare) Task=`9d9b66af-9a1b-49c9-ad31-193c8daac3ae`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-99::DEBUG::2014-10-23 17:14:28,287::task::595::Storage.TaskManager.Task::(_updateState) Task=`9d9b66af-9a1b-49c9-ad31-193c8daac3ae`::moving from state preparing -> state finished >Thread-99::DEBUG::2014-10-23 17:14:28,287::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-99::DEBUG::2014-10-23 17:14:28,287::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-99::DEBUG::2014-10-23 17:14:28,287::task::993::Storage.TaskManager.Task::(_decref) Task=`9d9b66af-9a1b-49c9-ad31-193c8daac3ae`::ref 0 aborting False >Thread-99::DEBUG::2014-10-23 17:14:28,288::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-99::DEBUG::2014-10-23 17:14:28,288::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:29,290::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:29,291::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-100::DEBUG::2014-10-23 17:14:29,291::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-100::DEBUG::2014-10-23 17:14:29,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`176402ae-02b0-4aa3-bc41-54feee86499c`::moving from state init -> state preparing >Thread-100::INFO::2014-10-23 17:14:29,292::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-100::DEBUG::2014-10-23 17:14:29,292::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-100::DEBUG::2014-10-23 17:14:29,292::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-100::INFO::2014-10-23 17:14:29,292::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-100::DEBUG::2014-10-23 17:14:29,292::task::1191::Storage.TaskManager.Task::(prepare) Task=`176402ae-02b0-4aa3-bc41-54feee86499c`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-100::DEBUG::2014-10-23 17:14:29,292::task::595::Storage.TaskManager.Task::(_updateState) Task=`176402ae-02b0-4aa3-bc41-54feee86499c`::moving from state preparing -> state finished >Thread-100::DEBUG::2014-10-23 17:14:29,292::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-100::DEBUG::2014-10-23 17:14:29,292::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-100::DEBUG::2014-10-23 17:14:29,292::task::993::Storage.TaskManager.Task::(_decref) Task=`176402ae-02b0-4aa3-bc41-54feee86499c`::ref 0 aborting False >Thread-100::DEBUG::2014-10-23 17:14:29,292::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-100::DEBUG::2014-10-23 17:14:29,293::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:29,315::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:29,315::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-101::DEBUG::2014-10-23 17:14:29,316::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:30,300::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:30,301::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-102::DEBUG::2014-10-23 17:14:30,301::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-102::DEBUG::2014-10-23 17:14:30,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`1073fc78-b023-4e18-b3fa-344a2c8505f2`::moving from state init -> state preparing >Thread-102::INFO::2014-10-23 17:14:30,302::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-102::DEBUG::2014-10-23 17:14:30,302::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-102::DEBUG::2014-10-23 17:14:30,302::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-102::INFO::2014-10-23 17:14:30,302::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-102::DEBUG::2014-10-23 17:14:30,302::task::1191::Storage.TaskManager.Task::(prepare) Task=`1073fc78-b023-4e18-b3fa-344a2c8505f2`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-102::DEBUG::2014-10-23 17:14:30,302::task::595::Storage.TaskManager.Task::(_updateState) Task=`1073fc78-b023-4e18-b3fa-344a2c8505f2`::moving from state preparing -> state finished >Thread-102::DEBUG::2014-10-23 17:14:30,302::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-102::DEBUG::2014-10-23 17:14:30,302::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-102::DEBUG::2014-10-23 17:14:30,302::task::993::Storage.TaskManager.Task::(_decref) Task=`1073fc78-b023-4e18-b3fa-344a2c8505f2`::ref 0 aborting False >Thread-102::DEBUG::2014-10-23 17:14:30,303::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-102::DEBUG::2014-10-23 17:14:30,303::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:30,544::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:30,544::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-103::DEBUG::2014-10-23 17:14:30,544::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getDeviceList' in bridge with {u'storageType': 3} >Thread-103::DEBUG::2014-10-23 17:14:30,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f1f60a4-99a9-4b99-82aa-7563de3fd23c`::moving from state init -> state preparing >Thread-103::INFO::2014-10-23 17:14:30,545::logUtils::44::dispatcher::(wrapper) Run and protect: getDeviceList(storageType=3, options={}) >Thread-103::DEBUG::2014-10-23 17:14:30,545::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-103::DEBUG::2014-10-23 17:14:30,545::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-103::DEBUG::2014-10-23 17:14:30,546::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-103::DEBUG::2014-10-23 17:14:30,546::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-103::DEBUG::2014-10-23 17:14:30,546::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-103::DEBUG::2014-10-23 17:14:30,546::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-103::DEBUG::2014-10-23 17:14:30,563::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-103::DEBUG::2014-10-23 17:14:30,563::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-103::DEBUG::2014-10-23 17:14:30,631::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-103::DEBUG::2014-10-23 17:14:30,631::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,631::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,632::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,632::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,632::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,632::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,632::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-103::DEBUG::2014-10-23 17:14:30,632::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,633::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size (cwd None) >Thread-103::DEBUG::2014-10-23 17:14:30,736::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-103::DEBUG::2014-10-23 17:14:30,736::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-103::DEBUG::2014-10-23 17:14:30,752::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --test --metadatasize 128m --metadatacopies 2 --metadataignore y /dev/mapper/1cmg02 /dev/mapper/1cmg01 (cwd None) >Thread-103::DEBUG::2014-10-23 17:14:30,863::lvm::288::Storage.Misc.excCmd::(cmd) FAILED: <err> = ' TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.\n Can\'t initialize physical volume "/dev/mapper/1cmg01" of volume group "5c8c20ff-41e1-4a43-91a8-fbffa2f8334f" without -ff\n'; <rc> = 5 >Thread-103::DEBUG::2014-10-23 17:14:30,864::lvm::852::Storage.LVM::(testPVCreate) rc: 5, out: [' Physical volume "/dev/mapper/1cmg02" successfully created'], err: [' TEST MODE: Metadata will NOT be updated and volumes will not be (de)activated.', ' Can\'t initialize physical volume "/dev/mapper/1cmg01" of volume group "5c8c20ff-41e1-4a43-91a8-fbffa2f8334f" without -ff'], unusedDevs: set(['/dev/mapper/1cmg02']), usedDevs: set(['/dev/mapper/1cmg01']) >Thread-103::INFO::2014-10-23 17:14:30,865::logUtils::47::dispatcher::(wrapper) Run and protect: getDeviceList, Return response: {'devList': [{'status': 'free', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'initiatorname': 'default', 'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK'}, {'status': 'used', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'fwrev': '0001', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'initiatorname': 'default', 'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'password': '******', 'port': '3260'}], 'logicalblocksize': '512', 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'devtype': 'iSCSI', 'physicalblocksize': '512', 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK'}]} >Thread-103::DEBUG::2014-10-23 17:14:30,865::task::1191::Storage.TaskManager.Task::(prepare) Task=`5f1f60a4-99a9-4b99-82aa-7563de3fd23c`::finished: {'devList': [{'status': 'free', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ'}, {'status': 'used', 'fwrev': '0001', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'pathstatus': [{'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC'}]} >Thread-103::DEBUG::2014-10-23 17:14:30,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`5f1f60a4-99a9-4b99-82aa-7563de3fd23c`::moving from state preparing -> state finished >Thread-103::DEBUG::2014-10-23 17:14:30,865::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-103::DEBUG::2014-10-23 17:14:30,865::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-103::DEBUG::2014-10-23 17:14:30,865::task::993::Storage.TaskManager.Task::(_decref) Task=`5f1f60a4-99a9-4b99-82aa-7563de3fd23c`::ref 0 aborting False >Thread-103::WARNING::2014-10-23 17:14:30,865::Bridge::212::root::(_typeFixup) Fixing up type BlockDevicePathInfo >Thread-103::WARNING::2014-10-23 17:14:30,866::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-103::WARNING::2014-10-23 17:14:30,866::Bridge::212::root::(_typeFixup) Fixing up type BlockDevicePathInfo >Thread-103::WARNING::2014-10-23 17:14:30,867::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-103::DEBUG::2014-10-23 17:14:30,867::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getDeviceList' in bridge with [{'status': 'free', 'fwrev': '0001', 'vgUUID': '', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '102005473280', 'pathstatus': [{'deviceType': 'iSCSI', 'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'pvUUID': '50oiMG-ajls-1zsU-HWhE-D7oG-HEaR-hDFNNZ'}, {'status': 'used', 'fwrev': '0001', 'vgUUID': 'L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'logicalblocksize': '512', 'devtype': 'iSCSI', 'physicalblocksize': '512', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg01', 'productID': 'VIRTUAL-DISK', 'vendorID': 'RH-str01', 'capacity': '91268055040', 'pathstatus': [{'deviceType': 'iSCSI', 'physdev': 'sdb', 'type': 'iSCSI', 'state': 'active', 'lun': '1'}], 'pvUUID': '0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC'}] >Thread-103::DEBUG::2014-10-23 17:14:30,867::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:31,306::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:31,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-104::DEBUG::2014-10-23 17:14:31,307::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-104::DEBUG::2014-10-23 17:14:31,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9e77ca8-7993-4d10-84ac-835c36f8a47c`::moving from state init -> state preparing >Thread-104::INFO::2014-10-23 17:14:31,308::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-104::DEBUG::2014-10-23 17:14:31,308::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-104::DEBUG::2014-10-23 17:14:31,308::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-104::INFO::2014-10-23 17:14:31,308::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-104::DEBUG::2014-10-23 17:14:31,308::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9e77ca8-7993-4d10-84ac-835c36f8a47c`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-104::DEBUG::2014-10-23 17:14:31,308::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9e77ca8-7993-4d10-84ac-835c36f8a47c`::moving from state preparing -> state finished >Thread-104::DEBUG::2014-10-23 17:14:31,308::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-104::DEBUG::2014-10-23 17:14:31,309::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-104::DEBUG::2014-10-23 17:14:31,309::task::993::Storage.TaskManager.Task::(_decref) Task=`b9e77ca8-7993-4d10-84ac-835c36f8a47c`::ref 0 aborting False >Thread-104::DEBUG::2014-10-23 17:14:31,309::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-104::DEBUG::2014-10-23 17:14:31,309::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:32,312::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-105::DEBUG::2014-10-23 17:14:32,313::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >JsonRpcServer::DEBUG::2014-10-23 17:14:32,313::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-105::DEBUG::2014-10-23 17:14:32,314::task::595::Storage.TaskManager.Task::(_updateState) Task=`180473f5-b565-4b00-8508-975ee42b1034`::moving from state init -> state preparing >Thread-105::INFO::2014-10-23 17:14:32,314::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-105::DEBUG::2014-10-23 17:14:32,314::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-105::DEBUG::2014-10-23 17:14:32,314::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-105::INFO::2014-10-23 17:14:32,314::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-105::DEBUG::2014-10-23 17:14:32,315::task::1191::Storage.TaskManager.Task::(prepare) Task=`180473f5-b565-4b00-8508-975ee42b1034`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-105::DEBUG::2014-10-23 17:14:32,315::task::595::Storage.TaskManager.Task::(_updateState) Task=`180473f5-b565-4b00-8508-975ee42b1034`::moving from state preparing -> state finished >Thread-105::DEBUG::2014-10-23 17:14:32,315::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-105::DEBUG::2014-10-23 17:14:32,315::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-105::DEBUG::2014-10-23 17:14:32,315::task::993::Storage.TaskManager.Task::(_decref) Task=`180473f5-b565-4b00-8508-975ee42b1034`::ref 0 aborting False >Thread-105::DEBUG::2014-10-23 17:14:32,315::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-105::DEBUG::2014-10-23 17:14:32,315::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:32,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:32,369::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-106::DEBUG::2014-10-23 17:14:32,370::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:33,318::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:33,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-107::DEBUG::2014-10-23 17:14:33,319::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-107::DEBUG::2014-10-23 17:14:33,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`f576b02e-1c4a-4266-864f-63b6a8e7c666`::moving from state init -> state preparing >Thread-107::INFO::2014-10-23 17:14:33,320::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-107::DEBUG::2014-10-23 17:14:33,320::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-107::DEBUG::2014-10-23 17:14:33,320::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-107::INFO::2014-10-23 17:14:33,320::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-107::DEBUG::2014-10-23 17:14:33,320::task::1191::Storage.TaskManager.Task::(prepare) Task=`f576b02e-1c4a-4266-864f-63b6a8e7c666`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-107::DEBUG::2014-10-23 17:14:33,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`f576b02e-1c4a-4266-864f-63b6a8e7c666`::moving from state preparing -> state finished >Thread-107::DEBUG::2014-10-23 17:14:33,321::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-107::DEBUG::2014-10-23 17:14:33,321::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-107::DEBUG::2014-10-23 17:14:33,321::task::993::Storage.TaskManager.Task::(_decref) Task=`f576b02e-1c4a-4266-864f-63b6a8e7c666`::ref 0 aborting False >Thread-107::DEBUG::2014-10-23 17:14:33,321::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-107::DEBUG::2014-10-23 17:14:33,321::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:34,324::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:34,325::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-108::DEBUG::2014-10-23 17:14:34,325::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-108::DEBUG::2014-10-23 17:14:34,326::task::595::Storage.TaskManager.Task::(_updateState) Task=`77ea4512-fbf1-4734-b3a8-5a1612d6e080`::moving from state init -> state preparing >Thread-108::INFO::2014-10-23 17:14:34,326::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-108::DEBUG::2014-10-23 17:14:34,326::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-108::DEBUG::2014-10-23 17:14:34,326::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-108::INFO::2014-10-23 17:14:34,326::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-108::DEBUG::2014-10-23 17:14:34,326::task::1191::Storage.TaskManager.Task::(prepare) Task=`77ea4512-fbf1-4734-b3a8-5a1612d6e080`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-108::DEBUG::2014-10-23 17:14:34,326::task::595::Storage.TaskManager.Task::(_updateState) Task=`77ea4512-fbf1-4734-b3a8-5a1612d6e080`::moving from state preparing -> state finished >Thread-108::DEBUG::2014-10-23 17:14:34,326::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-108::DEBUG::2014-10-23 17:14:34,326::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-108::DEBUG::2014-10-23 17:14:34,326::task::993::Storage.TaskManager.Task::(_decref) Task=`77ea4512-fbf1-4734-b3a8-5a1612d6e080`::ref 0 aborting False >Thread-108::DEBUG::2014-10-23 17:14:34,326::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-108::DEBUG::2014-10-23 17:14:34,327::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:35,345::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:35,346::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-109::DEBUG::2014-10-23 17:14:35,346::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-109::DEBUG::2014-10-23 17:14:35,346::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fb1d73f-913f-4c6f-bb14-6631622a89e9`::moving from state init -> state preparing >Thread-109::INFO::2014-10-23 17:14:35,346::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-109::DEBUG::2014-10-23 17:14:35,347::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-109::DEBUG::2014-10-23 17:14:35,347::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-109::INFO::2014-10-23 17:14:35,347::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-109::DEBUG::2014-10-23 17:14:35,347::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fb1d73f-913f-4c6f-bb14-6631622a89e9`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-109::DEBUG::2014-10-23 17:14:35,347::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fb1d73f-913f-4c6f-bb14-6631622a89e9`::moving from state preparing -> state finished >Thread-109::DEBUG::2014-10-23 17:14:35,347::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-109::DEBUG::2014-10-23 17:14:35,347::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-109::DEBUG::2014-10-23 17:14:35,347::task::993::Storage.TaskManager.Task::(_decref) Task=`7fb1d73f-913f-4c6f-bb14-6631622a89e9`::ref 0 aborting False >Thread-109::DEBUG::2014-10-23 17:14:35,347::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-109::DEBUG::2014-10-23 17:14:35,347::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:35,394::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:35,395::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-110::DEBUG::2014-10-23 17:14:35,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`f030e7c3-58bd-4fc3-95fb-00ba7c36f697`::moving from state init -> state preparing >Thread-110::INFO::2014-10-23 17:14:35,396::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-110::INFO::2014-10-23 17:14:35,396::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00039765', 'lastCheck': '19.4', 'valid': True}} >Thread-110::DEBUG::2014-10-23 17:14:35,396::task::1191::Storage.TaskManager.Task::(prepare) Task=`f030e7c3-58bd-4fc3-95fb-00ba7c36f697`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.00039765', 'lastCheck': '19.4', 'valid': True}} >Thread-110::DEBUG::2014-10-23 17:14:35,396::task::595::Storage.TaskManager.Task::(_updateState) Task=`f030e7c3-58bd-4fc3-95fb-00ba7c36f697`::moving from state preparing -> state finished >Thread-110::DEBUG::2014-10-23 17:14:35,396::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-110::DEBUG::2014-10-23 17:14:35,396::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-110::DEBUG::2014-10-23 17:14:35,397::task::993::Storage.TaskManager.Task::(_decref) Task=`f030e7c3-58bd-4fc3-95fb-00ba7c36f697`::ref 0 aborting False >Thread-110::DEBUG::2014-10-23 17:14:35,398::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:35,493::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:35,494::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-111::DEBUG::2014-10-23 17:14:35,495::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:36,350::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:36,351::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-112::DEBUG::2014-10-23 17:14:36,351::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-112::DEBUG::2014-10-23 17:14:36,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`70fc9e9f-d022-427b-b118-ed2bbe9d7fa7`::moving from state init -> state preparing >Thread-112::INFO::2014-10-23 17:14:36,352::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-112::DEBUG::2014-10-23 17:14:36,352::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-112::DEBUG::2014-10-23 17:14:36,352::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-112::INFO::2014-10-23 17:14:36,352::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-112::DEBUG::2014-10-23 17:14:36,352::task::1191::Storage.TaskManager.Task::(prepare) Task=`70fc9e9f-d022-427b-b118-ed2bbe9d7fa7`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-112::DEBUG::2014-10-23 17:14:36,352::task::595::Storage.TaskManager.Task::(_updateState) Task=`70fc9e9f-d022-427b-b118-ed2bbe9d7fa7`::moving from state preparing -> state finished >Thread-112::DEBUG::2014-10-23 17:14:36,353::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-112::DEBUG::2014-10-23 17:14:36,353::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-112::DEBUG::2014-10-23 17:14:36,353::task::993::Storage.TaskManager.Task::(_decref) Task=`70fc9e9f-d022-427b-b118-ed2bbe9d7fa7`::ref 0 aborting False >Thread-112::DEBUG::2014-10-23 17:14:36,353::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-112::DEBUG::2014-10-23 17:14:36,353::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,217::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,217::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >1c557727-9394-4848-9699-e6a94a5708d6::WARNING::2014-10-23 17:14:37,218::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,218::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >1c557727-9394-4848-9699-e6a94a5708d6::WARNING::2014-10-23 17:14:37,218::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,219::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,219::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,219::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,219::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,219::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,220::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000388995 s, 5.3 MB/s\n'; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000388995 s, 5.3 MB/s'], size: 2048 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,224::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,225::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,225::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,225::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,225::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,225::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,318::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,318::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,318::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:37,319::clusterlock::295::Storage.SANLock::(acquire) Acquiring cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:37,356::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:37,357::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-113::DEBUG::2014-10-23 17:14:37,357::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-113::DEBUG::2014-10-23 17:14:37,358::task::595::Storage.TaskManager.Task::(_updateState) Task=`0645760e-0091-4973-8d45-682bf4272d4e`::moving from state init -> state preparing >Thread-113::INFO::2014-10-23 17:14:37,358::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-113::DEBUG::2014-10-23 17:14:37,358::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-113::DEBUG::2014-10-23 17:14:37,358::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-113::INFO::2014-10-23 17:14:37,358::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-113::DEBUG::2014-10-23 17:14:37,359::task::1191::Storage.TaskManager.Task::(prepare) Task=`0645760e-0091-4973-8d45-682bf4272d4e`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-113::DEBUG::2014-10-23 17:14:37,359::task::595::Storage.TaskManager.Task::(_updateState) Task=`0645760e-0091-4973-8d45-682bf4272d4e`::moving from state preparing -> state finished >Thread-113::DEBUG::2014-10-23 17:14:37,359::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-113::DEBUG::2014-10-23 17:14:37,359::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-113::DEBUG::2014-10-23 17:14:37,359::task::993::Storage.TaskManager.Task::(_decref) Task=`0645760e-0091-4973-8d45-682bf4272d4e`::ref 0 aborting False >Thread-113::DEBUG::2014-10-23 17:14:37,359::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-113::DEBUG::2014-10-23 17:14:37,359::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,380::clusterlock::321::Storage.SANLock::(acquire) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,380::sp::272::Storage.StoragePool::(startSpm) spm lock acquired successfully >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,380::spbackends::422::Storage.StoragePoolMemoryBackend::(setSpmStatus) this storage pool implementation ignores the set spm status requests (lver=0, spmid=1) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,381::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`f62cb5e6-e8bf-4423-bdac-122ef8f2f473`::Request was made in '/usr/share/vdsm/storage/sp.py' line '426' at '_upgradePool' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,381::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,381::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,381::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`f62cb5e6-e8bf-4423-bdac-122ef8f2f473`::Granted request >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:37,381::sp::432::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,382::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`ecc24708-01c8-480a-be03-7cc17a69e411`::Request was made in '/usr/share/vdsm/storage/sp.py' line '434' at '_upgradePool' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,382::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,382::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,382::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`ecc24708-01c8-480a-be03-7cc17a69e411`::Granted request >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,382::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,383::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,387::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.00026292 s, 7.8 MB/s\n'; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.00026292 s, 7.8 MB/s'], size: 2048 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,388::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,389::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,389::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,389::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,486::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,486::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,486::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:37,487::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:37,487::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >1c557727-9394-4848-9699-e6a94a5708d6::INFO::2014-10-23 17:14:37,487::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,487::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,487::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,487::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,487::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::sp::437::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::sp::444::Storage.StoragePool::(_upgradePool) Registering with state change event >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::sp::447::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' (0 active users) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067', Clearing records. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,488::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,489::blockSD::1126::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /sbin/fsck -p /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,530::blockSD::1126::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,530::blockSD::1151::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /sbin/tune2fs -j /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,551::blockSD::1151::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,552::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t ext3 /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,634::blockSD::1163::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,652::blockSD::1163::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,652::sd::647::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master/vms >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,653::__init__::159::IOProcessClient::(_run) Starting IOProcess... >Thread-114::DEBUG::2014-10-23 17:14:37,657::__init__::232::IOProcess::(_processLogs) Closing unrelated FDs... >Thread-114::DEBUG::2014-10-23 17:14:37,657::__init__::232::IOProcess::(_processLogs) Opening communication channels... >Thread-114::DEBUG::2014-10-23 17:14:37,657::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,658::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,658::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,658::__init__::232::IOProcess::(_processLogs) (1) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,658::__init__::232::IOProcess::(_processLogs) (1) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,659::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,659::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,659::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,659::__init__::232::IOProcess::(_processLogs) (2) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,659::__init__::232::IOProcess::(_processLogs) (2) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,660::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,660::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,660::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,661::__init__::232::IOProcess::(_processLogs) (3) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,661::__init__::232::IOProcess::(_processLogs) (3) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,661::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,661::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,662::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,662::__init__::232::IOProcess::(_processLogs) (4) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,662::__init__::232::IOProcess::(_processLogs) (4) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,662::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,662::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,663::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,663::__init__::232::IOProcess::(_processLogs) (5) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,663::__init__::232::IOProcess::(_processLogs) (5) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,663::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,664::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,664::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,664::__init__::232::IOProcess::(_processLogs) (6) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,664::__init__::232::IOProcess::(_processLogs) (6) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,664::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,665::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,665::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,665::__init__::232::IOProcess::(_processLogs) (7) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,665::__init__::232::IOProcess::(_processLogs) (7) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,665::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,666::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,666::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,666::__init__::232::IOProcess::(_processLogs) (8) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,668::__init__::232::IOProcess::(_processLogs) (8) Queuing response >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,668::sd::650::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master/tasks >Thread-114::DEBUG::2014-10-23 17:14:37,668::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,669::__init__::232::IOProcess::(_processLogs) (9) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,669::__init__::232::IOProcess::(_processLogs) (9) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,670::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,670::__init__::232::IOProcess::(_processLogs) (10) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,670::__init__::232::IOProcess::(_processLogs) (10) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,671::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,671::__init__::232::IOProcess::(_processLogs) (11) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,671::__init__::232::IOProcess::(_processLogs) (11) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,671::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,672::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,672::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,672::__init__::232::IOProcess::(_processLogs) (12) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,672::__init__::232::IOProcess::(_processLogs) (12) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,673::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,673::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,673::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,673::__init__::232::IOProcess::(_processLogs) (13) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,673::__init__::232::IOProcess::(_processLogs) (13) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,674::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,674::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,674::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,674::__init__::232::IOProcess::(_processLogs) (14) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,674::__init__::232::IOProcess::(_processLogs) (14) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,675::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,675::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,675::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,675::__init__::232::IOProcess::(_processLogs) (15) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,676::__init__::232::IOProcess::(_processLogs) (15) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:37,676::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:37,676::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:37,676::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:37,676::__init__::232::IOProcess::(_processLogs) (16) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:14:37,677::__init__::232::IOProcess::(_processLogs) (16) Queuing response >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,678::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,680::storage_mailbox::574::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1'] >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,680::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000 (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,697::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129234 s, 79.2 MB/s\n'; <rc> = 0 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,698::storage_mailbox::582::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool 00000002-0002-0002-0002-000000000067 >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,698::sp::319::Storage.StoragePool::(startSpm) SPM mailbox ready for pool 00000002-0002-0002-0002-000000000067 on master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Dummy-120::DEBUG::2014-10-23 17:14:37,699::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,699::sp::327::Storage.StoragePool::(startSpm) ended. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,704::task::595::Storage.TaskManager.Task::(_updateState) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::moving from state running -> state finished >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,704::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,704::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::task::916::Storage.TaskManager.Task::(_runJobs) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::Task.run: exit - success: result >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,705::task::993::Storage.TaskManager.Task::(_decref) Task=`1c557727-9394-4848-9699-e6a94a5708d6`::ref 0 aborting False >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:14:37,706::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 0 >Dummy-120::DEBUG::2014-10-23 17:14:37,745::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125063 s, 81.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,362::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:38,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-121::DEBUG::2014-10-23 17:14:38,363::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-121::DEBUG::2014-10-23 17:14:38,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`494a2b16-44ed-445b-bc79-bfa10f795ca7`::moving from state init -> state preparing >Thread-121::INFO::2014-10-23 17:14:38,364::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-121::DEBUG::2014-10-23 17:14:38,364::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-121::DEBUG::2014-10-23 17:14:38,364::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-121::INFO::2014-10-23 17:14:38,364::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-121::DEBUG::2014-10-23 17:14:38,365::task::1191::Storage.TaskManager.Task::(prepare) Task=`494a2b16-44ed-445b-bc79-bfa10f795ca7`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'}} >Thread-121::DEBUG::2014-10-23 17:14:38,365::task::595::Storage.TaskManager.Task::(_updateState) Task=`494a2b16-44ed-445b-bc79-bfa10f795ca7`::moving from state preparing -> state finished >Thread-121::DEBUG::2014-10-23 17:14:38,365::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-121::DEBUG::2014-10-23 17:14:38,365::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-121::DEBUG::2014-10-23 17:14:38,365::task::993::Storage.TaskManager.Task::(_decref) Task=`494a2b16-44ed-445b-bc79-bfa10f795ca7`::ref 0 aborting False >Thread-121::DEBUG::2014-10-23 17:14:38,365::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': '1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-121::DEBUG::2014-10-23 17:14:38,365::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-122::DEBUG::2014-10-23 17:14:38,369::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:14:38,369::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-122::DEBUG::2014-10-23 17:14:38,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f806c73-f306-4c13-b4dd-099fcf9ae5d6`::moving from state init -> state preparing >Thread-122::INFO::2014-10-23 17:14:38,370::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-122::INFO::2014-10-23 17:14:38,381::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-122::DEBUG::2014-10-23 17:14:38,381::task::1191::Storage.TaskManager.Task::(prepare) Task=`3f806c73-f306-4c13-b4dd-099fcf9ae5d6`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-122::DEBUG::2014-10-23 17:14:38,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`3f806c73-f306-4c13-b4dd-099fcf9ae5d6`::moving from state preparing -> state finished >Thread-122::DEBUG::2014-10-23 17:14:38,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-122::DEBUG::2014-10-23 17:14:38,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-122::DEBUG::2014-10-23 17:14:38,382::task::993::Storage.TaskManager.Task::(_decref) Task=`3f806c73-f306-4c13-b4dd-099fcf9ae5d6`::ref 0 aborting False >Thread-122::DEBUG::2014-10-23 17:14:38,382::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-122::DEBUG::2014-10-23 17:14:38,382::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,391::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:38,392::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-123::DEBUG::2014-10-23 17:14:38,392::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.clear' in bridge with {u'taskID': u'1c557727-9394-4848-9699-e6a94a5708d6'} >Thread-123::DEBUG::2014-10-23 17:14:38,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`7533ca13-3123-4488-901a-e106b595146b`::moving from state init -> state preparing >Thread-123::INFO::2014-10-23 17:14:38,393::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID=u'1c557727-9394-4848-9699-e6a94a5708d6', spUUID=None, options=None) >Thread-123::DEBUG::2014-10-23 17:14:38,393::taskManager::171::Storage.TaskManager::(clearTask) Entry. taskID: 1c557727-9394-4848-9699-e6a94a5708d6 >Thread-123::DEBUG::2014-10-23 17:14:38,393::taskManager::176::Storage.TaskManager::(clearTask) Return. >Thread-123::INFO::2014-10-23 17:14:38,393::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-123::DEBUG::2014-10-23 17:14:38,393::task::1191::Storage.TaskManager.Task::(prepare) Task=`7533ca13-3123-4488-901a-e106b595146b`::finished: None >Thread-123::DEBUG::2014-10-23 17:14:38,393::task::595::Storage.TaskManager.Task::(_updateState) Task=`7533ca13-3123-4488-901a-e106b595146b`::moving from state preparing -> state finished >Thread-123::DEBUG::2014-10-23 17:14:38,393::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-123::DEBUG::2014-10-23 17:14:38,393::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-123::DEBUG::2014-10-23 17:14:38,394::task::993::Storage.TaskManager.Task::(_decref) Task=`7533ca13-3123-4488-901a-e106b595146b`::ref 0 aborting False >Thread-123::DEBUG::2014-10-23 17:14:38,394::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.clear' in bridge with True >Thread-123::DEBUG::2014-10-23 17:14:38,394::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Detector thread::DEBUG::2014-10-23 17:14:38,582::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.25:44590 >Detector thread::DEBUG::2014-10-23 17:14:38,583::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.25:44590 >Detector thread::DEBUG::2014-10-23 17:14:38,583::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.34.63.25:44590 >Detector thread::DEBUG::2014-10-23 17:14:38,583::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-23 17:14:38,583::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-23 17:14:38,584::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.34.63.25', 44590) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,584::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,585::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,605::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:38,606::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-124::DEBUG::2014-10-23 17:14:38,607::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,625::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:38,626::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-125::DEBUG::2014-10-23 17:14:38,626::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.activate' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'storagedomainID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'} >Thread-125::DEBUG::2014-10-23 17:14:38,626::task::595::Storage.TaskManager.Task::(_updateState) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::moving from state init -> state preparing >Thread-125::INFO::2014-10-23 17:14:38,626::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-125::DEBUG::2014-10-23 17:14:38,627::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`44513965-0c3d-4635-93a3-eda70ef1e522`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1249' at 'activateStorageDomain' >Thread-125::DEBUG::2014-10-23 17:14:38,627::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-125::DEBUG::2014-10-23 17:14:38,627::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-125::DEBUG::2014-10-23 17:14:38,627::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`44513965-0c3d-4635-93a3-eda70ef1e522`::Granted request >Thread-125::DEBUG::2014-10-23 17:14:38,627::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-125::DEBUG::2014-10-23 17:14:38,627::task::993::Storage.TaskManager.Task::(_decref) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::ref 1 aborting False >Thread-125::DEBUG::2014-10-23 17:14:38,628::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`5d2fbf7c-e9f6-4791-9f99-4d719592e90f`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1250' at 'activateStorageDomain' >Thread-125::DEBUG::2014-10-23 17:14:38,628::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >Thread-125::DEBUG::2014-10-23 17:14:38,628::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >Thread-125::DEBUG::2014-10-23 17:14:38,628::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`5d2fbf7c-e9f6-4791-9f99-4d719592e90f`::Granted request >Thread-125::DEBUG::2014-10-23 17:14:38,628::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::_resourcesAcquired: Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (exclusive) >Thread-125::DEBUG::2014-10-23 17:14:38,628::task::993::Storage.TaskManager.Task::(_decref) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::ref 1 aborting False >Thread-125::INFO::2014-10-23 17:14:38,628::sp::1031::Storage.StoragePool::(activateSD) sdUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f spUUID=00000002-0002-0002-0002-000000000067 >Thread-125::INFO::2014-10-23 17:14:38,629::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-125::DEBUG::2014-10-23 17:14:38,629::task::1191::Storage.TaskManager.Task::(prepare) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::finished: None >Thread-125::DEBUG::2014-10-23 17:14:38,629::task::595::Storage.TaskManager.Task::(_updateState) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::moving from state preparing -> state finished >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': < ResourceRef 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >Thread-125::DEBUG::2014-10-23 17:14:38,629::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-125::DEBUG::2014-10-23 17:14:38,630::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-125::DEBUG::2014-10-23 17:14:38,630::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-125::DEBUG::2014-10-23 17:14:38,630::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-125::DEBUG::2014-10-23 17:14:38,630::task::993::Storage.TaskManager.Task::(_decref) Task=`5195a60c-42c8-4777-9e40-727ab3d16f72`::ref 0 aborting False >Thread-125::DEBUG::2014-10-23 17:14:38,630::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.activate' in bridge with True >Thread-125::DEBUG::2014-10-23 17:14:38,630::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:38,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:38,701::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-126::DEBUG::2014-10-23 17:14:38,701::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-126::DEBUG::2014-10-23 17:14:38,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`a789cc87-332d-45ee-b0a0-dd373c922847`::moving from state init -> state preparing >Thread-126::INFO::2014-10-23 17:14:38,702::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, options=None) >Thread-126::DEBUG::2014-10-23 17:14:38,702::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`9d97be3f-ef14-47a6-9bc0-98045bcde362`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-126::DEBUG::2014-10-23 17:14:38,702::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-126::DEBUG::2014-10-23 17:14:38,702::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-126::DEBUG::2014-10-23 17:14:38,702::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`9d97be3f-ef14-47a6-9bc0-98045bcde362`::Granted request >Thread-126::DEBUG::2014-10-23 17:14:38,703::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`48a3fe2d-55ff-498a-82d6-c4f838794fd3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-126::DEBUG::2014-10-23 17:14:38,703::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-126::DEBUG::2014-10-23 17:14:38,703::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-126::DEBUG::2014-10-23 17:14:38,703::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`48a3fe2d-55ff-498a-82d6-c4f838794fd3`::Granted request >Thread-126::DEBUG::2014-10-23 17:14:38,703::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'} >Thread-126::INFO::2014-10-23 17:14:38,703::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active'} >Thread-126::DEBUG::2014-10-23 17:14:38,703::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,703::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,704::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,704::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,704::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,704::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,704::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-126::DEBUG::2014-10-23 17:14:38,704::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-126::DEBUG::2014-10-23 17:14:38,704::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-126::DEBUG::2014-10-23 17:14:38,704::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-126::DEBUG::2014-10-23 17:14:38,704::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-126::DEBUG::2014-10-23 17:14:38,705::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:38,717::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-126::DEBUG::2014-10-23 17:14:38,717::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:38,809::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,810::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,811::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-126::DEBUG::2014-10-23 17:14:38,811::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,812::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:38,922::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-126::DEBUG::2014-10-23 17:14:38,922::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,923::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-126::DEBUG::2014-10-23 17:14:38,923::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:38,928::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000588613 s, 3.5 MB/s\n'; <rc> = 0 >Thread-126::DEBUG::2014-10-23 17:14:38,928::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000588613 s, 3.5 MB/s'], size: 2048 >Thread-126::DEBUG::2014-10-23 17:14:38,928::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-126::DEBUG::2014-10-23 17:14:38,928::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-126::DEBUG::2014-10-23 17:14:38,928::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-126::DEBUG::2014-10-23 17:14:38,928::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,929::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,929::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,929::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,929::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:38,929::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:39,021::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-126::DEBUG::2014-10-23 17:14:39,021::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:39,021::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-126::INFO::2014-10-23 17:14:39,022::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-126::INFO::2014-10-23 17:14:39,022::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-126::INFO::2014-10-23 17:14:39,022::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-126::DEBUG::2014-10-23 17:14:39,022::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-126::DEBUG::2014-10-23 17:14:39,022::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-126::DEBUG::2014-10-23 17:14:39,022::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-126::DEBUG::2014-10-23 17:14:39,022::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-126::DEBUG::2014-10-23 17:14:39,023::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-126::WARNING::2014-10-23 17:14:39,023::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-126::DEBUG::2014-10-23 17:14:39,023::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:39,023::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-126::DEBUG::2014-10-23 17:14:39,117::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-126::DEBUG::2014-10-23 17:14:39,117::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-126::DEBUG::2014-10-23 17:14:39,118::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-126::WARNING::2014-10-23 17:14:39,118::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-126::DEBUG::2014-10-23 17:14:39,118::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-126::WARNING::2014-10-23 17:14:39,118::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-126::INFO::2014-10-23 17:14:39,118::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-126::DEBUG::2014-10-23 17:14:39,118::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-126::INFO::2014-10-23 17:14:39,118::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-126::DEBUG::2014-10-23 17:14:39,118::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-126::DEBUG::2014-10-23 17:14:39,119::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-126::INFO::2014-10-23 17:14:39,120::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-126::DEBUG::2014-10-23 17:14:39,120::task::1191::Storage.TaskManager.Task::(prepare) Task=`a789cc87-332d-45ee-b0a0-dd373c922847`::finished: True >Thread-126::DEBUG::2014-10-23 17:14:39,120::task::595::Storage.TaskManager.Task::(_updateState) Task=`a789cc87-332d-45ee-b0a0-dd373c922847`::moving from state preparing -> state finished >Thread-126::DEBUG::2014-10-23 17:14:39,120::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-126::DEBUG::2014-10-23 17:14:39,120::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-126::DEBUG::2014-10-23 17:14:39,120::task::993::Storage.TaskManager.Task::(_decref) Task=`a789cc87-332d-45ee-b0a0-dd373c922847`::ref 0 aborting False >Thread-126::DEBUG::2014-10-23 17:14:39,120::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-126::DEBUG::2014-10-23 17:14:39,120::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:14:39,756::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:39,798::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123728 s, 82.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:41,626::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:41,628::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-127::DEBUG::2014-10-23 17:14:41,631::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:14:41,807::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:41,856::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0193514 s, 52.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:42,058::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:42,059::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-128::DEBUG::2014-10-23 17:14:42,059::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.create' in bridge with {u'devlist': [u'1cmg02'], u'force': False, u'name': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-128::DEBUG::2014-10-23 17:14:42,060::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c277bff-44c5-46e5-818c-5269d70ad96a`::moving from state init -> state preparing >Thread-128::INFO::2014-10-23 17:14:42,060::logUtils::44::dispatcher::(wrapper) Run and protect: createVG(vgname=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', devlist=[u'1cmg02'], force=False, options=None) >Thread-128::DEBUG::2014-10-23 17:14:42,061::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --metadatasize 128m --metadatacopies 2 --metadataignore y /dev/mapper/1cmg02 (cwd None) >Thread-128::DEBUG::2014-10-23 17:14:42,162::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-128::DEBUG::2014-10-23 17:14:42,163::lvm::483::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,163::lvm::486::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,163::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --metadataignore n /dev/mapper/1cmg02 (cwd None) >Thread-128::DEBUG::2014-10-23 17:14:42,268::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-128::DEBUG::2014-10-23 17:14:42,268::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --physicalextentsize 128m --addtag RHAT_storage_domain_UNREADY dae2b061-ea53-4ca2-8b91-f24f0641dc3f /dev/mapper/1cmg02 (cwd None) >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::483::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::486::Storage.OperationMutex::(_invalidatepvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::923::Storage.LVM::(createVG) Cache after createvg {'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': VG(uuid='L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', name='5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', attr=VG_ATTR(permission='w', resizeable='z', exported='-', partial='-', allocation='n', clustered='-'), size='90865401856', free='86704652288', extent_size='134217728', extent_count='677', free_count='646', tags=('MDT_LEASETIMESEC=60', 'MDT_PV0=pv:1cmg01&44&uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC&44&pestart:0&44&pecount:677&44&mapoffset:0', 'MDT_VERSION=3', 'MDT_TYPE=ISCSI', 'MDT_DESCRIPTION=iscsi', 'MDT_SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'MDT_LOGBLKSIZE=512', 'MDT_LEASERETRIES=3', 'MDT_IOOPTIMEOUTSEC=10', 'MDT_LOCKRENEWALINTERVALSEC=5', 'MDT_VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'MDT_PHYBLKSIZE=512', 'MDT_CLASS=Data', 'MDT_LOCKPOLICY=', 'RHAT_storage_domain', 'MDT_POOL_SPM_ID=-1', 'MDT_POOL_DESCRIPTION=Default', 'MDT_POOL_SPM_LVER=-1', 'MDT_POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MDT_MASTER_VERSION=1', 'MDT_ROLE=Master', 'MDT__SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'MDT_POOL_UUID=00000002-0002-0002-0002-000000000067'), vg_mda_size='134217728', vg_mda_free='67104256', lv_count='6', pv_count='1', pv_name=('/dev/mapper/1cmg01',), writeable=True, partial='OK'), u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': Stub(name=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', stale=True)} >Thread-128::DEBUG::2014-10-23 17:14:42,429::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-128::DEBUG::2014-10-23 17:14:42,430::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-128::DEBUG::2014-10-23 17:14:42,529::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-128::DEBUG::2014-10-23 17:14:42,529::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-128::INFO::2014-10-23 17:14:42,529::logUtils::47::dispatcher::(wrapper) Run and protect: createVG, Return response: {'uuid': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe'} >Thread-128::DEBUG::2014-10-23 17:14:42,529::task::1191::Storage.TaskManager.Task::(prepare) Task=`3c277bff-44c5-46e5-818c-5269d70ad96a`::finished: {'uuid': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe'} >Thread-128::DEBUG::2014-10-23 17:14:42,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`3c277bff-44c5-46e5-818c-5269d70ad96a`::moving from state preparing -> state finished >Thread-128::DEBUG::2014-10-23 17:14:42,530::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-128::DEBUG::2014-10-23 17:14:42,530::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-128::DEBUG::2014-10-23 17:14:42,530::task::993::Storage.TaskManager.Task::(_decref) Task=`3c277bff-44c5-46e5-818c-5269d70ad96a`::ref 0 aborting False >Thread-128::DEBUG::2014-10-23 17:14:42,530::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.create' in bridge with dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe >Thread-128::DEBUG::2014-10-23 17:14:42,530::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:42,534::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:42,535::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-129::DEBUG::2014-10-23 17:14:42,535::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.create' in bridge with {u'name': u'iscsi_1', u'domainType': 3, u'domainClass': 1, u'typeArgs': u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', u'version': u'3', u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-129::DEBUG::2014-10-23 17:14:42,536::task::595::Storage.TaskManager.Task::(_updateState) Task=`cda6ac2e-19d4-4067-8a6e-a6df0cd3decb`::moving from state init -> state preparing >Thread-129::INFO::2014-10-23 17:14:42,536::logUtils::44::dispatcher::(wrapper) Run and protect: createStorageDomain(storageType=3, sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', domainName=u'iscsi_1', typeSpecificArg=u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', domClass=1, domVersion=u'3', options=None) >Thread-129::ERROR::2014-10-23 17:14:42,537::sdc::137::Storage.StorageDomainCache::(_findDomain) looking for unfetched domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-129::ERROR::2014-10-23 17:14:42,537::sdc::154::Storage.StorageDomainCache::(_findUnfetchedDomain) looking for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-129::ERROR::2014-10-23 17:14:42,537::sdc::143::Storage.StorageDomainCache::(_findDomain) domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f not found >Traceback (most recent call last): > File "/usr/share/vdsm/storage/sdc.py", line 141, in _findDomain > dom = findMethod(sdUUID) > File "/usr/share/vdsm/storage/sdc.py", line 171, in _findUnfetchedDomain > raise se.StorageDomainDoesNotExist(sdUUID) >StorageDomainDoesNotExist: Storage domain does not exist: (u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f',) >Thread-129::INFO::2014-10-23 17:14:42,537::blockSD::484::Storage.StorageDomain::(create) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f domainName=iscsi_1 domClass=1 vgUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe storageType=3 version=3 >Thread-129::DEBUG::2014-10-23 17:14:42,538::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,538::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:42,627::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:42,627::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-129::DEBUG::2014-10-23 17:14:42,627::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-129::INFO::2014-10-23 17:14:42,628::blockSD::468::Storage.StorageDomain::(metaSize) size 512 MB (metaratio 262144) >Thread-129::DEBUG::2014-10-23 17:14:42,628::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 512m --name metadata dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:42,752::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:42,752::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,752::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,752::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,753::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,753::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,753::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:42,848::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:42,848::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-129::DEBUG::2014-10-23 17:14:42,848::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,848::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,849::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:42,942::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:42,943::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-129::INFO::2014-10-23 17:14:42,943::blockSD::653::Storage.StorageDomain::(getMetaDataMapping) Create: SORT MAPPING: ['/dev/mapper/1cmg02'] >Thread-129::DEBUG::2014-10-23 17:14:42,943::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:42,943::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size /dev/mapper/1cmg02 (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,041::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,042::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,042::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 16m --name outbox dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,153::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,153::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,153::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,154::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,154::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,154::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 2048m --name leases dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,270::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,270::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,271::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,271::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,271::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,271::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/leases (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,280::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,280::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 8m --name ids dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,395::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,396::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,396::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,396::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,396::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,396::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/ids (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,406::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,407::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 16m --name inbox dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,527::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,527::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,527::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,527::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,527::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,528::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvcreate --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --contiguous n --size 1024m --name master dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,645::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ' WARNING: This metadata update is NOT backed up\n WARNING: This metadata update is NOT backed up\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,646::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,646::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,646::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,646::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:43,646::lvm::1072::Storage.Misc.excCmd::(createLV) /usr/bin/sudo -n /bin/chown vdsm:qemu /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/master (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:43,655::lvm::1072::Storage.Misc.excCmd::(createLV) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:43,655::blockSD::1342::Storage.Misc.excCmd::(_createVMSfs) /usr/bin/sudo -n /sbin/mkfs -q -j -E nodiscard /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/master (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:43,980::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:44,110::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0943241 s, 10.9 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:44,607::blockSD::1342::Storage.Misc.excCmd::(_createVMSfs) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:44,608::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available n dae2b061-ea53-4ca2-8b91-f24f0641dc3f/master (cwd None) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:44,680::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:44,682::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-130::DEBUG::2014-10-23 17:14:44,683::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-129::DEBUG::2014-10-23 17:14:44,712::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:44,712::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:44,712::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:44,712::blockSD::535::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata bs=1048576 seek=0 skip=0 conv=notrunc count=40 oflag=direct (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,112::blockSD::535::Storage.Misc.excCmd::(create) SUCCESS: <err> = '40+0 records in\n40+0 records out\n41943040 bytes (42 MB) copied, 0.394192 s, 106 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,112::misc::262::Storage.Misc::(validateDDBytes) err: ['40+0 records in', '40+0 records out', '41943040 bytes (42 MB) copied, 0.394192 s, 106 MB/s'], size: 41943040 >Thread-129::DEBUG::2014-10-23 17:14:45,112::blockSD::537::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/inbox bs=1024000 seek=0 skip=0 conv=notrunc count=1 oflag=direct (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,127::blockSD::537::Storage.Misc.excCmd::(create) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0099471 s, 103 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,128::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '1024000 bytes (1.0 MB) copied, 0.0099471 s, 103 MB/s'], size: 1024000 >Thread-129::DEBUG::2014-10-23 17:14:45,128::blockSD::539::Storage.Misc.excCmd::(create) /bin/nice -n 19 /usr/bin/ionice -c 3 /bin/dd if=/dev/zero of=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/outbox bs=1024000 seek=0 skip=0 conv=notrunc count=1 oflag=direct (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,143::blockSD::539::Storage.Misc.excCmd::(create) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.00988436 s, 104 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,143::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '1024000 bytes (1.0 MB) copied, 0.00988436 s, 104 MB/s'], size: 1024000 >Thread-129::DEBUG::2014-10-23 17:14:45,143::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-129::DEBUG::2014-10-23 17:14:45,143::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,144::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,240::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,240::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,241::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,336::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,337::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,337::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=[] >Thread-129::DEBUG::2014-10-23 17:14:45,337::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-129::DEBUG::2014-10-23 17:14:45,337::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-129::DEBUG::2014-10-23 17:14:45,337::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-129::DEBUG::2014-10-23 17:14:45,337::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', u'DESCRIPTION=iscsi_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'PHYBLKSIZE=512', 'POOL_UUID=', 'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'ROLE=Regular', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475'] >Thread-129::DEBUG::2014-10-23 17:14:45,337::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,337::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,337::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,338::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,338::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,338::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,432::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,432::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,433::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_PHYBLKSIZE=512, MDT_DESCRIPTION=iscsi_1, MDT_POOL_UUID=, MDT_LEASERETRIES=3, MDT_CLASS=Data, MDT_IOOPTIMEOUTSEC=10, MDT_LOCKRENEWALINTERVALSEC=5, MDT_LOGBLKSIZE=512, MDT_LEASETIMESEC=60, MDT_SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f, MDT_VERSION=3, MDT_PV0=pv:1cmg02&44&uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr&44&pestart:0&44&pecount:757&44&mapoffset:0, MDT_VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe, MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475, MDT_TYPE=ISCSI, MDT_ROLE=Regular, MDT_LOCKPOLICY= removing= >Thread-129::DEBUG::2014-10-23 17:14:45,433::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --addtag MDT_LEASETIMESEC=60 --addtag MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475 --addtag MDT_POOL_UUID= --addtag MDT_VERSION=3 --addtag MDT_TYPE=ISCSI --addtag MDT_LOGBLKSIZE=512 --addtag MDT_VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe --addtag MDT_LEASERETRIES=3 --addtag MDT_IOOPTIMEOUTSEC=10 --addtag MDT_LOCKRENEWALINTERVALSEC=5 --addtag MDT_LOCKPOLICY= --addtag MDT_ROLE=Regular --addtag MDT_PHYBLKSIZE=512 --addtag 'MDT_PV0=pv:1cmg02&44&uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr&44&pestart:0&44&pecount:757&44&mapoffset:0' --addtag MDT_SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f --addtag MDT_CLASS=Data --addtag MDT_DESCRIPTION=iscsi_1 dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,606::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,607::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,607::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,607::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-129::DEBUG::2014-10-23 17:14:45,607::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag RHAT_storage_domain_UNREADY --addtag RHAT_storage_domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,766::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,766::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,766::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,766::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-129::DEBUG::2014-10-23 17:14:45,767::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,772::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000548498 s, 3.7 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,772::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000548498 s, 3.7 MB/s'], size: 2048 >Thread-129::DEBUG::2014-10-23 17:14:45,772::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-129::DEBUG::2014-10-23 17:14:45,772::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-129::DEBUG::2014-10-23 17:14:45,772::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-129::DEBUG::2014-10-23 17:14:45,772::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,773::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,773::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,773::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,773::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,773::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:45,867::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:45,868::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:45,868::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1'] >Thread-129::DEBUG::2014-10-23 17:14:45,868::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --autobackup n --available y dae2b061-ea53-4ca2-8b91-f24f0641dc3f/master (cwd None) >Thread-129::DEBUG::2014-10-23 17:14:46,001::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:46,001::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:46,001::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-129::DEBUG::2014-10-23 17:14:46,001::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS' >Thread-129::DEBUG::2014-10-23 17:14:46,002::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS' >Thread-129::DEBUG::2014-10-23 17:14:46,002::resourceManager::421::Storage.ResourceManager::(registerNamespace) Registering namespace 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS' >Thread-129::DEBUG::2014-10-23 17:14:46,002::clusterlock::173::Storage.initSANLock::(initSANLock) Initializing SANLock for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Dummy-120::DEBUG::2014-10-23 17:14:46,123::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:46,164::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123717 s, 82.8 MB/s\n'; <rc> = 0 >Thread-129::DEBUG::2014-10-23 17:14:46,311::sd::443::Storage.StorageDomain::(initSPMlease) lease initialized successfully >Thread-129::DEBUG::2014-10-23 17:14:46,311::hsm::2691::Storage.HSM::(createStorageDomain) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain, dae2b061-ea53-4ca2-8b91-f24f0641dc3f: storage.blockSD.findDomain} >Thread-129::INFO::2014-10-23 17:14:46,311::logUtils::47::dispatcher::(wrapper) Run and protect: createStorageDomain, Return response: None >Thread-129::DEBUG::2014-10-23 17:14:46,312::task::1191::Storage.TaskManager.Task::(prepare) Task=`cda6ac2e-19d4-4067-8a6e-a6df0cd3decb`::finished: None >Thread-129::DEBUG::2014-10-23 17:14:46,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`cda6ac2e-19d4-4067-8a6e-a6df0cd3decb`::moving from state preparing -> state finished >Thread-129::DEBUG::2014-10-23 17:14:46,312::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-129::DEBUG::2014-10-23 17:14:46,312::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-129::DEBUG::2014-10-23 17:14:46,312::task::993::Storage.TaskManager.Task::(_decref) Task=`cda6ac2e-19d4-4067-8a6e-a6df0cd3decb`::ref 0 aborting False >Thread-129::DEBUG::2014-10-23 17:14:46,312::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.create' in bridge with True >Thread-129::DEBUG::2014-10-23 17:14:46,313::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:46,319::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:46,320::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-131::DEBUG::2014-10-23 17:14:46,322::task::595::Storage.TaskManager.Task::(_updateState) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::moving from state init -> state preparing >Thread-131::INFO::2014-10-23 17:14:46,322::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainStats(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', options=None) >Thread-131::DEBUG::2014-10-23 17:14:46,322::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`8eb1fe01-ae1e-4306-8e60-70af1c9bf6e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2830' at 'getStorageDomainStats' >Thread-131::DEBUG::2014-10-23 17:14:46,322::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'shared' >Thread-131::DEBUG::2014-10-23 17:14:46,323::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'shared' (1 active user) >Thread-131::DEBUG::2014-10-23 17:14:46,323::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`8eb1fe01-ae1e-4306-8e60-70af1c9bf6e7`::Granted request >Thread-131::DEBUG::2014-10-23 17:14:46,323::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (shared) >Thread-131::DEBUG::2014-10-23 17:14:46,323::task::993::Storage.TaskManager.Task::(_decref) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::ref 1 aborting False >Thread-131::DEBUG::2014-10-23 17:14:46,323::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images >Thread-131::DEBUG::2014-10-23 17:14:46,323::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/metadata >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/leases to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/leases >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/ids to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/ids >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/inbox to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/inbox >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/outbox to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/outbox >Thread-131::DEBUG::2014-10-23 17:14:46,324::blockSD::1308::Storage.StorageDomain::(refreshDirTree) Creating symlink from /dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/master to /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md/master >Thread-131::DEBUG::2014-10-23 17:14:46,324::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-131::DEBUG::2014-10-23 17:14:46,325::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-131::DEBUG::2014-10-23 17:14:46,325::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-131::DEBUG::2014-10-23 17:14:46,325::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-131::DEBUG::2014-10-23 17:14:46,325::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-131::DEBUG::2014-10-23 17:14:46,325::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-131::DEBUG::2014-10-23 17:14:46,330::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000263721 s, 7.8 MB/s\n'; <rc> = 0 >Thread-131::DEBUG::2014-10-23 17:14:46,330::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000263721 s, 7.8 MB/s'], size: 2048 >Thread-131::DEBUG::2014-10-23 17:14:46,331::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-131::DEBUG::2014-10-23 17:14:46,331::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-131::DEBUG::2014-10-23 17:14:46,331::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-131::DEBUG::2014-10-23 17:14:46,331::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-131::DEBUG::2014-10-23 17:14:46,331::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-131::DEBUG::2014-10-23 17:14:46,428::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-131::DEBUG::2014-10-23 17:14:46,428::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-131::INFO::2014-10-23 17:14:46,428::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainStats, Return response: {'stats': {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '97442070528', 'disktotal': '101602820096', 'mdafree': '67104768'}} >Thread-131::DEBUG::2014-10-23 17:14:46,429::task::1191::Storage.TaskManager.Task::(prepare) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::finished: {'stats': {'mdasize': '134217728', 'mdathreshold': True, 'mdavalid': True, 'diskfree': '97442070528', 'disktotal': '101602820096', 'mdafree': '67104768'}} >Thread-131::DEBUG::2014-10-23 17:14:46,429::task::595::Storage.TaskManager.Task::(_updateState) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::moving from state preparing -> state finished >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>} >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-131::DEBUG::2014-10-23 17:14:46,429::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-131::DEBUG::2014-10-23 17:14:46,430::task::993::Storage.TaskManager.Task::(_decref) Task=`99c864f5-2be6-412d-a844-40358fa811ca`::ref 0 aborting False >Thread-131::DEBUG::2014-10-23 17:14:46,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:46,481::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:46,482::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-132::DEBUG::2014-10-23 17:14:46,482::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.getInfo' in bridge with {u'lvmvolumegroupID': u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe'} >Thread-132::DEBUG::2014-10-23 17:14:46,483::task::595::Storage.TaskManager.Task::(_updateState) Task=`079a8774-9222-4984-982c-33db8dbe554a`::moving from state init -> state preparing >Thread-132::INFO::2014-10-23 17:14:46,483::logUtils::44::dispatcher::(wrapper) Run and protect: getVGInfo(vgUUID=u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', options=None) >Thread-132::INFO::2014-10-23 17:14:46,510::logUtils::47::dispatcher::(wrapper) Run and protect: getVGInfo, Return response: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-132::DEBUG::2014-10-23 17:14:46,510::task::1191::Storage.TaskManager.Task::(prepare) Task=`079a8774-9222-4984-982c-33db8dbe554a`::finished: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-132::DEBUG::2014-10-23 17:14:46,510::task::595::Storage.TaskManager.Task::(_updateState) Task=`079a8774-9222-4984-982c-33db8dbe554a`::moving from state preparing -> state finished >Thread-132::DEBUG::2014-10-23 17:14:46,510::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-132::DEBUG::2014-10-23 17:14:46,510::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-132::DEBUG::2014-10-23 17:14:46,510::task::993::Storage.TaskManager.Task::(_decref) Task=`079a8774-9222-4984-982c-33db8dbe554a`::ref 0 aborting False >Thread-132::WARNING::2014-10-23 17:14:46,510::Bridge::212::root::(_typeFixup) Fixing up type VolumeGroupInfo >Thread-132::WARNING::2014-10-23 17:14:46,510::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-132::DEBUG::2014-10-23 17:14:46,511::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.getInfo' in bridge with {'state': 'OK', 'deviceType': 3, 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}} >Thread-132::DEBUG::2014-10-23 17:14:46,511::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:46,921::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:46,922::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-133::DEBUG::2014-10-23 17:14:46,922::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000000-0000-0000-0000-000000000000', u'domainType': 3} >Thread-133::DEBUG::2014-10-23 17:14:46,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`1248f4d0-452e-4f65-afad-d462bd80e089`::moving from state init -> state preparing >Thread-133::INFO::2014-10-23 17:14:46,923::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000000-0000-0000-0000-000000000000', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-133::DEBUG::2014-10-23 17:14:46,923::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-133::DEBUG::2014-10-23 17:14:46,934::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-133::DEBUG::2014-10-23 17:14:46,935::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-133::DEBUG::2014-10-23 17:14:46,944::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-133::DEBUG::2014-10-23 17:14:46,945::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-133::DEBUG::2014-10-23 17:14:46,953::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-133::DEBUG::2014-10-23 17:14:46,954::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f') >Thread-133::DEBUG::2014-10-23 17:14:46,954::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain, dae2b061-ea53-4ca2-8b91-f24f0641dc3f: storage.blockSD.findDomain} >Thread-133::INFO::2014-10-23 17:14:46,954::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-133::DEBUG::2014-10-23 17:14:46,954::task::1191::Storage.TaskManager.Task::(prepare) Task=`1248f4d0-452e-4f65-afad-d462bd80e089`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-133::DEBUG::2014-10-23 17:14:46,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`1248f4d0-452e-4f65-afad-d462bd80e089`::moving from state preparing -> state finished >Thread-133::DEBUG::2014-10-23 17:14:46,954::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-133::DEBUG::2014-10-23 17:14:46,954::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-133::DEBUG::2014-10-23 17:14:46,954::task::993::Storage.TaskManager.Task::(_decref) Task=`1248f4d0-452e-4f65-afad-d462bd80e089`::ref 0 aborting False >Thread-133::DEBUG::2014-10-23 17:14:46,955::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-133::DEBUG::2014-10-23 17:14:46,955::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:46,961::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:46,962::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-134::DEBUG::2014-10-23 17:14:46,962::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.getInfo' in bridge with {u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-134::DEBUG::2014-10-23 17:14:46,963::task::595::Storage.TaskManager.Task::(_updateState) Task=`713188cb-03a8-440d-b04d-f93846e10928`::moving from state init -> state preparing >Thread-134::INFO::2014-10-23 17:14:46,963::logUtils::44::dispatcher::(wrapper) Run and protect: getStorageDomainInfo(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', options=None) >Thread-134::DEBUG::2014-10-23 17:14:46,963::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-134::DEBUG::2014-10-23 17:14:46,964::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-134::DEBUG::2014-10-23 17:14:46,964::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-134::DEBUG::2014-10-23 17:14:46,964::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-134::DEBUG::2014-10-23 17:14:46,964::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-134::DEBUG::2014-10-23 17:14:46,964::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:46,977::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-134::DEBUG::2014-10-23 17:14:46,977::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,081::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,082::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-134::DEBUG::2014-10-23 17:14:47,082::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,084::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,181::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,181::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,181::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-134::DEBUG::2014-10-23 17:14:47,181::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,186::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000514617 s, 4.0 MB/s\n'; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,187::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000514617 s, 4.0 MB/s'], size: 2048 >Thread-134::DEBUG::2014-10-23 17:14:47,187::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-134::DEBUG::2014-10-23 17:14:47,187::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-134::DEBUG::2014-10-23 17:14:47,187::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-134::DEBUG::2014-10-23 17:14:47,187::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,187::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,187::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,188::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,188::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,188::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:47,219::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-72::DEBUG::2014-10-23 17:14:47,220::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,285::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,286::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1'] >Thread-134::INFO::2014-10-23 17:14:47,286::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS already registered >Thread-134::INFO::2014-10-23 17:14:47,287::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS already registered >Thread-134::INFO::2014-10-23 17:14:47,287::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS already registered >Thread-134::INFO::2014-10-23 17:14:47,287::blockSD::839::Storage.StorageDomain::(validate) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-134::DEBUG::2014-10-23 17:14:47,287::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:47,319::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:47,320::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,320::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-72::DEBUG::2014-10-23 17:14:47,321::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:47,326::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000408816 s, 5.0 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:47,326::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000408816 s, 5.0 MB/s'], size: 2048 >Thread-72::DEBUG::2014-10-23 17:14:47,326::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-72::DEBUG::2014-10-23 17:14:47,326::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-72::DEBUG::2014-10-23 17:14:47,326::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-72::DEBUG::2014-10-23 17:14:47,326::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,326::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,327::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,327::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,327::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,327::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,381::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,382::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Thread-72::DEBUG::2014-10-23 17:14:47,424::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:47,425::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,425::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-134::DEBUG::2014-10-23 17:14:47,426::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,427::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-72::INFO::2014-10-23 17:14:47,428::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-72::INFO::2014-10-23 17:14:47,429::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-134::DEBUG::2014-10-23 17:14:47,429::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-72::INFO::2014-10-23 17:14:47,430::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-134::DEBUG::2014-10-23 17:14:47,431::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-72::DEBUG::2014-10-23 17:14:47,431::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-134::DEBUG::2014-10-23 17:14:47,436::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,436::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:47,528::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:14:47,528::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:47,533::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000334268 s, 12.3 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:14:47,535::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:47,535::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:47,535::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:47,535::__init__::232::IOProcess::(_processLogs) (17) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:14:47,536::__init__::232::IOProcess::(_processLogs) (17) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:47,536::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:47,537::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:47,537::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:47,537::__init__::232::IOProcess::(_processLogs) (18) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:14:47,538::__init__::232::IOProcess::(_processLogs) (18) Queuing response >Thread-134::DEBUG::2014-10-23 17:14:47,538::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-134::DEBUG::2014-10-23 17:14:47,539::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-134::DEBUG::2014-10-23 17:14:47,539::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1'] >Thread-134::DEBUG::2014-10-23 17:14:47,539::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`a9fb7ab6-2676-4205-99a6-c5ece9884e73`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2812' at 'getStorageDomainInfo' >Thread-134::DEBUG::2014-10-23 17:14:47,539::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'shared' >Thread-134::DEBUG::2014-10-23 17:14:47,540::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'shared' (1 active user) >Thread-134::DEBUG::2014-10-23 17:14:47,540::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`a9fb7ab6-2676-4205-99a6-c5ece9884e73`::Granted request >Thread-134::DEBUG::2014-10-23 17:14:47,540::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`713188cb-03a8-440d-b04d-f93846e10928`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (shared) >Thread-134::DEBUG::2014-10-23 17:14:47,540::task::993::Storage.TaskManager.Task::(_decref) Task=`713188cb-03a8-440d-b04d-f93846e10928`::ref 1 aborting False >Thread-134::INFO::2014-10-23 17:14:47,540::logUtils::47::dispatcher::(wrapper) Run and protect: getStorageDomainInfo, Return response: {'info': {'uuid': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vguuid': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'state': 'OK', 'version': '3', 'role': 'Regular', 'type': 'ISCSI', 'class': 'Data', 'pool': [], 'name': 'iscsi_1'}} >Thread-134::DEBUG::2014-10-23 17:14:47,540::task::1191::Storage.TaskManager.Task::(prepare) Task=`713188cb-03a8-440d-b04d-f93846e10928`::finished: {'info': {'uuid': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vguuid': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'state': 'OK', 'version': '3', 'role': 'Regular', 'type': 'ISCSI', 'class': 'Data', 'pool': [], 'name': 'iscsi_1'}} >Thread-134::DEBUG::2014-10-23 17:14:47,540::task::595::Storage.TaskManager.Task::(_updateState) Task=`713188cb-03a8-440d-b04d-f93846e10928`::moving from state preparing -> state finished >Thread-134::DEBUG::2014-10-23 17:14:47,540::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>} >Thread-134::DEBUG::2014-10-23 17:14:47,541::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-134::DEBUG::2014-10-23 17:14:47,541::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-134::DEBUG::2014-10-23 17:14:47,541::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-134::DEBUG::2014-10-23 17:14:47,541::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-134::DEBUG::2014-10-23 17:14:47,541::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-134::DEBUG::2014-10-23 17:14:47,541::task::993::Storage.TaskManager.Task::(_decref) Task=`713188cb-03a8-440d-b04d-f93846e10928`::ref 0 aborting False >Thread-134::WARNING::2014-10-23 17:14:47,541::Bridge::212::root::(_typeFixup) Fixing up type StorageDomainInfo >Thread-134::WARNING::2014-10-23 17:14:47,541::Bridge::467::root::(fieldClone) fieldClone: class -> domainClass >Thread-134::DEBUG::2014-10-23 17:14:47,542::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.getInfo' in bridge with {'uuid': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'domainClass': 'Data', 'vguuid': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'state': 'OK', 'version': '3', 'role': 'Regular', 'type': 'ISCSI', 'class': 'Data', 'pool': [], 'name': 'iscsi_1'} >Thread-134::DEBUG::2014-10-23 17:14:47,542::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:47,597::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:47,598::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-135::DEBUG::2014-10-23 17:14:47,598::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.attach' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-135::DEBUG::2014-10-23 17:14:47,599::task::595::Storage.TaskManager.Task::(_updateState) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::moving from state init -> state preparing >Thread-135::INFO::2014-10-23 17:14:47,599::logUtils::44::dispatcher::(wrapper) Run and protect: attachStorageDomain(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-135::DEBUG::2014-10-23 17:14:47,599::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`24b83366-0c29-4b8f-bcc6-a57cca7c9f3e`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1190' at 'attachStorageDomain' >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`24b83366-0c29-4b8f-bcc6-a57cca7c9f3e`::Granted request >Thread-135::DEBUG::2014-10-23 17:14:47,600::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-135::DEBUG::2014-10-23 17:14:47,600::task::993::Storage.TaskManager.Task::(_decref) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::ref 1 aborting False >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`99ffd0ce-c5db-45fd-bf73-e6b8e2c39eb1`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1191' at 'attachStorageDomain' >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'exclusive' >Thread-135::DEBUG::2014-10-23 17:14:47,600::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'exclusive' (1 active user) >Thread-135::DEBUG::2014-10-23 17:14:47,601::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`99ffd0ce-c5db-45fd-bf73-e6b8e2c39eb1`::Granted request >Thread-135::DEBUG::2014-10-23 17:14:47,601::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (exclusive) >Thread-135::DEBUG::2014-10-23 17:14:47,601::task::993::Storage.TaskManager.Task::(_decref) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::ref 1 aborting False >Thread-135::INFO::2014-10-23 17:14:47,601::sp::876::Storage.StoragePool::(attachSD) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f spUUID=00000002-0002-0002-0002-000000000067 >Thread-135::INFO::2014-10-23 17:14:47,601::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:47,703::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:47,704::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-136::DEBUG::2014-10-23 17:14:47,705::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:14:48,173::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:48,214::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123584 s, 82.9 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:14:50,223::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:50,263::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123744 s, 82.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:50,731::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:50,732::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-137::DEBUG::2014-10-23 17:14:50,733::task::595::Storage.TaskManager.Task::(_updateState) Task=`48c05bfb-17f3-40d7-a0da-f787d4092459`::moving from state init -> state preparing >Thread-137::INFO::2014-10-23 17:14:50,734::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-137::INFO::2014-10-23 17:14:50,734::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000334268', 'lastCheck': '3.2', 'valid': True}} >Thread-137::DEBUG::2014-10-23 17:14:50,734::task::1191::Storage.TaskManager.Task::(prepare) Task=`48c05bfb-17f3-40d7-a0da-f787d4092459`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000334268', 'lastCheck': '3.2', 'valid': True}} >Thread-137::DEBUG::2014-10-23 17:14:50,734::task::595::Storage.TaskManager.Task::(_updateState) Task=`48c05bfb-17f3-40d7-a0da-f787d4092459`::moving from state preparing -> state finished >Thread-137::DEBUG::2014-10-23 17:14:50,734::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-137::DEBUG::2014-10-23 17:14:50,734::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-137::DEBUG::2014-10-23 17:14:50,734::task::993::Storage.TaskManager.Task::(_decref) Task=`48c05bfb-17f3-40d7-a0da-f787d4092459`::ref 0 aborting False >Thread-137::DEBUG::2014-10-23 17:14:50,736::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:50,825::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:50,826::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-138::DEBUG::2014-10-23 17:14:50,826::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:14:52,272::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:52,316::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126806 s, 80.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:54,089::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:54,090::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-139::DEBUG::2014-10-23 17:14:54,091::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:14:54,325::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:54,372::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0173017 s, 59.2 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:14:56,381::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:56,421::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0123102 s, 83.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:14:57,144::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:14:57,145::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-140::DEBUG::2014-10-23 17:14:57,146::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:14:57,539::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:14:57,544::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000521371 s, 7.9 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:14:57,546::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:57,546::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:57,546::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:57,546::__init__::232::IOProcess::(_processLogs) (19) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:14:57,547::__init__::232::IOProcess::(_processLogs) (19) Queuing response >Thread-114::DEBUG::2014-10-23 17:14:57,547::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:14:57,548::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:14:57,548::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:14:57,548::__init__::232::IOProcess::(_processLogs) (20) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:14:57,549::__init__::232::IOProcess::(_processLogs) (20) Queuing response >Dummy-120::DEBUG::2014-10-23 17:14:58,430::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:14:58,475::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0135188 s, 75.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:00,160::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:00,161::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-141::DEBUG::2014-10-23 17:15:00,162::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:00,485::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:00,534::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0201614 s, 50.8 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:15:02,543::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:02,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0218239 s, 46.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:03,184::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:03,185::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-142::DEBUG::2014-10-23 17:15:03,186::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:04,643::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:04,684::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122261 s, 83.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:06,212::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:06,213::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-143::DEBUG::2014-10-23 17:15:06,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb438ea-5065-4c7f-8b57-60d7c1aa9e9d`::moving from state init -> state preparing >Thread-143::INFO::2014-10-23 17:15:06,215::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-143::INFO::2014-10-23 17:15:06,215::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521371', 'lastCheck': '8.7', 'valid': True}} >Thread-143::DEBUG::2014-10-23 17:15:06,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`6cb438ea-5065-4c7f-8b57-60d7c1aa9e9d`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000521371', 'lastCheck': '8.7', 'valid': True}} >Thread-143::DEBUG::2014-10-23 17:15:06,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`6cb438ea-5065-4c7f-8b57-60d7c1aa9e9d`::moving from state preparing -> state finished >Thread-143::DEBUG::2014-10-23 17:15:06,215::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-143::DEBUG::2014-10-23 17:15:06,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-143::DEBUG::2014-10-23 17:15:06,215::task::993::Storage.TaskManager.Task::(_decref) Task=`6cb438ea-5065-4c7f-8b57-60d7c1aa9e9d`::ref 0 aborting False >Thread-143::DEBUG::2014-10-23 17:15:06,217::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:06,319::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:06,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-144::DEBUG::2014-10-23 17:15:06,320::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:06,694::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:06,734::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124324 s, 82.4 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:07,552::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:07,557::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000557801 s, 7.3 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:15:07,559::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:07,559::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:07,559::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:07,560::__init__::232::IOProcess::(_processLogs) (21) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:07,560::__init__::232::IOProcess::(_processLogs) (21) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:07,560::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:07,560::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:07,561::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:07,561::__init__::232::IOProcess::(_processLogs) (22) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:07,561::__init__::232::IOProcess::(_processLogs) (22) Queuing response >Thread-135::DEBUG::2014-10-23 17:15:08,602::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f successfully acquired (id: 1) >Thread-135::DEBUG::2014-10-23 17:15:08,603::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images >Thread-135::WARNING::2014-10-23 17:15:08,603::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images already exists >Thread-135::DEBUG::2014-10-23 17:15:08,603::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md >Thread-135::WARNING::2014-10-23 17:15:08,603::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md already exists >Thread-135::DEBUG::2014-10-23 17:15:08,603::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,603::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,603::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,604::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,604::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-135::DEBUG::2014-10-23 17:15:08,604::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:08,609::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000567777 s, 3.6 MB/s\n'; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:08,609::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000567777 s, 3.6 MB/s'], size: 2048 >Thread-135::DEBUG::2014-10-23 17:15:08,609::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-135::DEBUG::2014-10-23 17:15:08,609::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-135::DEBUG::2014-10-23 17:15:08,610::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,610::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:08,709::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:08,709::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,709::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1'] >Thread-135::INFO::2014-10-23 17:15:08,710::clusterlock::295::Storage.SANLock::(acquire) Acquiring cluster lock for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (id: 1) >Dummy-120::DEBUG::2014-10-23 17:15:08,743::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:08,787::clusterlock::321::Storage.SANLock::(acquire) Cluster lock for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f successfully acquired (id: 1) >Dummy-120::DEBUG::2014-10-23 17:15:08,788::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0116628 s, 87.8 MB/s\n'; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:08,790::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,791::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,792::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,793::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,794::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,795::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:08,893::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:08,894::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,894::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475', 'POOL_UUID=', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1'] >Thread-135::DEBUG::2014-10-23 17:15:08,894::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-135::DEBUG::2014-10-23 17:15:08,894::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-135::DEBUG::2014-10-23 17:15:08,895::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'PHYBLKSIZE=512', 'POOL_UUID=00000002-0002-0002-0002-000000000067', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'ROLE=Regular', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca'] >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,895::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:08,990::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:08,991::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:08,991::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca, MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 removing=MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475, MDT_POOL_UUID= >Thread-135::DEBUG::2014-10-23 17:15:08,991::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475 --deltag MDT_POOL_UUID= --addtag MDT__SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca --addtag MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-135::DEBUG::2014-10-23 17:15:09,171::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:09,171::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:09,171::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-135::DEBUG::2014-10-23 17:15:09,172::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-135::INFO::2014-10-23 17:15:09,172::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': 'Attached'} >Thread-135::DEBUG::2014-10-23 17:15:09,172::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images >Thread-135::WARNING::2014-10-23 17:15:09,172::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images already exists >Thread-135::DEBUG::2014-10-23 17:15:09,172::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md >Thread-135::WARNING::2014-10-23 17:15:09,172::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md already exists >Thread-135::INFO::2014-10-23 17:15:09,172::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f to /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-135::DEBUG::2014-10-23 17:15:09,173::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f to /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-135::INFO::2014-10-23 17:15:09,173::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-135::DEBUG::2014-10-23 17:15:09,216::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f successfully released >Thread-135::INFO::2014-10-23 17:15:09,216::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:10,358::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:10,359::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-145::DEBUG::2014-10-23 17:15:10,360::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:10,810::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:10,852::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125664 s, 81.5 MB/s\n'; <rc> = 0 >Thread-135::DEBUG::2014-10-23 17:15:11,217::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f released successfully (id: 1) >Thread-135::DEBUG::2014-10-23 17:15:11,217::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-135::DEBUG::2014-10-23 17:15:11,218::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-135::DEBUG::2014-10-23 17:15:11,219::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-135::INFO::2014-10-23 17:15:11,219::logUtils::47::dispatcher::(wrapper) Run and protect: attachStorageDomain, Return response: None >Thread-135::DEBUG::2014-10-23 17:15:11,220::task::1191::Storage.TaskManager.Task::(prepare) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::finished: None >Thread-135::DEBUG::2014-10-23 17:15:11,221::task::595::Storage.TaskManager.Task::(_updateState) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::moving from state preparing -> state finished >Thread-135::DEBUG::2014-10-23 17:15:11,221::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-135::DEBUG::2014-10-23 17:15:11,222::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-135::DEBUG::2014-10-23 17:15:11,223::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-135::DEBUG::2014-10-23 17:15:11,224::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-135::DEBUG::2014-10-23 17:15:11,225::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-135::DEBUG::2014-10-23 17:15:11,225::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-135::DEBUG::2014-10-23 17:15:11,226::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-135::DEBUG::2014-10-23 17:15:11,227::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-135::DEBUG::2014-10-23 17:15:11,228::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-135::DEBUG::2014-10-23 17:15:11,228::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-135::DEBUG::2014-10-23 17:15:11,229::task::993::Storage.TaskManager.Task::(_decref) Task=`feeabd9a-e1d0-47b5-b29a-5c5f46a5f92c`::ref 0 aborting False >Thread-135::DEBUG::2014-10-23 17:15:11,230::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.attach' in bridge with True >Thread-135::DEBUG::2014-10-23 17:15:11,231::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,248::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-146::DEBUG::2014-10-23 17:15:11,251::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,251::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-146::DEBUG::2014-10-23 17:15:11,253::task::595::Storage.TaskManager.Task::(_updateState) Task=`7640069a-a85d-4b9a-a928-9c0526c22d46`::moving from state init -> state preparing >Thread-146::INFO::2014-10-23 17:15:11,254::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-146::INFO::2014-10-23 17:15:11,265::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-146::DEBUG::2014-10-23 17:15:11,266::task::1191::Storage.TaskManager.Task::(prepare) Task=`7640069a-a85d-4b9a-a928-9c0526c22d46`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-146::DEBUG::2014-10-23 17:15:11,266::task::595::Storage.TaskManager.Task::(_updateState) Task=`7640069a-a85d-4b9a-a928-9c0526c22d46`::moving from state preparing -> state finished >Thread-146::DEBUG::2014-10-23 17:15:11,267::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-146::DEBUG::2014-10-23 17:15:11,267::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-146::DEBUG::2014-10-23 17:15:11,268::task::993::Storage.TaskManager.Task::(_decref) Task=`7640069a-a85d-4b9a-a928-9c0526c22d46`::ref 0 aborting False >Thread-146::DEBUG::2014-10-23 17:15:11,269::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-146::DEBUG::2014-10-23 17:15:11,270::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,283::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-147::DEBUG::2014-10-23 17:15:11,285::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,285::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-147::DEBUG::2014-10-23 17:15:11,288::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::moving from state init -> state preparing >Thread-147::INFO::2014-10-23 17:15:11,288::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-147::DEBUG::2014-10-23 17:15:11,290::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f8609bda-3617-4535-b08e-a2d85204fd97`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-147::DEBUG::2014-10-23 17:15:11,290::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-147::DEBUG::2014-10-23 17:15:11,291::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-147::DEBUG::2014-10-23 17:15:11,292::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f8609bda-3617-4535-b08e-a2d85204fd97`::Granted request >Thread-147::DEBUG::2014-10-23 17:15:11,293::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-147::DEBUG::2014-10-23 17:15:11,293::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::ref 1 aborting False >Thread-147::INFO::2014-10-23 17:15:11,305::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-147::DEBUG::2014-10-23 17:15:11,306::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-147::DEBUG::2014-10-23 17:15:11,307::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::moving from state preparing -> state finished >Thread-147::DEBUG::2014-10-23 17:15:11,307::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-147::DEBUG::2014-10-23 17:15:11,308::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-147::DEBUG::2014-10-23 17:15:11,309::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-147::DEBUG::2014-10-23 17:15:11,310::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-147::DEBUG::2014-10-23 17:15:11,310::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-147::DEBUG::2014-10-23 17:15:11,311::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-147::DEBUG::2014-10-23 17:15:11,312::task::993::Storage.TaskManager.Task::(_decref) Task=`b6b9a7a1-0592-4a72-b574-a973035acf18`::ref 0 aborting False >Thread-147::DEBUG::2014-10-23 17:15:11,313::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': 'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-147::DEBUG::2014-10-23 17:15:11,316::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,352::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-148::DEBUG::2014-10-23 17:15:11,354::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.getImages' in bridge with {u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,354::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-148::DEBUG::2014-10-23 17:15:11,357::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::moving from state init -> state preparing >Thread-148::INFO::2014-10-23 17:15:11,358::logUtils::44::dispatcher::(wrapper) Run and protect: getImagesList(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', options=None) >Thread-148::DEBUG::2014-10-23 17:15:11,359::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`faac45b3-beb6-4c15-bed9-17e82dcdb9e0`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '3304' at 'getImagesList' >Thread-148::DEBUG::2014-10-23 17:15:11,359::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'shared' >Thread-148::DEBUG::2014-10-23 17:15:11,360::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'shared' (1 active user) >Thread-148::DEBUG::2014-10-23 17:15:11,360::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`faac45b3-beb6-4c15-bed9-17e82dcdb9e0`::Granted request >Thread-148::DEBUG::2014-10-23 17:15:11,361::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (shared) >Thread-148::DEBUG::2014-10-23 17:15:11,362::task::993::Storage.TaskManager.Task::(_decref) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::ref 1 aborting False >Thread-148::DEBUG::2014-10-23 17:15:11,363::lvm::419::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' got the operation mutex >Thread-148::DEBUG::2014-10-23 17:15:11,363::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm lvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,vg_name,attr,size,seg_start_pe,devices,tags dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-148::DEBUG::2014-10-23 17:15:11,460::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-148::DEBUG::2014-10-23 17:15:11,461::lvm::454::Storage.LVM::(_reloadlvs) lvs reloaded >Thread-148::DEBUG::2014-10-23 17:15:11,462::lvm::454::Storage.OperationMutex::(_reloadlvs) Operation 'lvm reload operation' released the operation mutex >Thread-148::INFO::2014-10-23 17:15:11,463::logUtils::47::dispatcher::(wrapper) Run and protect: getImagesList, Return response: {'imageslist': []} >Thread-148::DEBUG::2014-10-23 17:15:11,464::task::1191::Storage.TaskManager.Task::(prepare) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::finished: {'imageslist': []} >Thread-148::DEBUG::2014-10-23 17:15:11,465::task::595::Storage.TaskManager.Task::(_updateState) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::moving from state preparing -> state finished >Thread-148::DEBUG::2014-10-23 17:15:11,465::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>} >Thread-148::DEBUG::2014-10-23 17:15:11,466::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-148::DEBUG::2014-10-23 17:15:11,467::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-148::DEBUG::2014-10-23 17:15:11,468::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-148::DEBUG::2014-10-23 17:15:11,468::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-148::DEBUG::2014-10-23 17:15:11,469::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-148::DEBUG::2014-10-23 17:15:11,470::task::993::Storage.TaskManager.Task::(_decref) Task=`b6f273c6-b3f8-4096-8b3c-41eb7c33dffa`::ref 0 aborting False >Thread-148::DEBUG::2014-10-23 17:15:11,471::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.getImages' in bridge with [] >Thread-148::DEBUG::2014-10-23 17:15:11,472::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,594::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-149::DEBUG::2014-10-23 17:15:11,597::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,597::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-149::DEBUG::2014-10-23 17:15:11,600::task::595::Storage.TaskManager.Task::(_updateState) Task=`d34eeb1f-9fac-4609-9109-a201bb9e294a`::moving from state init -> state preparing >Thread-149::INFO::2014-10-23 17:15:11,600::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-149::DEBUG::2014-10-23 17:15:11,602::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-149::DEBUG::2014-10-23 17:15:11,616::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-149::DEBUG::2014-10-23 17:15:11,618::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-149::DEBUG::2014-10-23 17:15:11,629::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-149::DEBUG::2014-10-23 17:15:11,630::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-149::DEBUG::2014-10-23 17:15:11,640::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-149::DEBUG::2014-10-23 17:15:11,641::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-149::DEBUG::2014-10-23 17:15:11,642::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-149::DEBUG::2014-10-23 17:15:11,747::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-149::DEBUG::2014-10-23 17:15:11,749::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-149::DEBUG::2014-10-23 17:15:11,749::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f') >Thread-149::DEBUG::2014-10-23 17:15:11,750::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain, dae2b061-ea53-4ca2-8b91-f24f0641dc3f: storage.blockSD.findDomain} >Thread-149::INFO::2014-10-23 17:15:11,751::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-149::DEBUG::2014-10-23 17:15:11,755::task::1191::Storage.TaskManager.Task::(prepare) Task=`d34eeb1f-9fac-4609-9109-a201bb9e294a`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-149::DEBUG::2014-10-23 17:15:11,756::task::595::Storage.TaskManager.Task::(_updateState) Task=`d34eeb1f-9fac-4609-9109-a201bb9e294a`::moving from state preparing -> state finished >Thread-149::DEBUG::2014-10-23 17:15:11,757::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-149::DEBUG::2014-10-23 17:15:11,757::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-149::DEBUG::2014-10-23 17:15:11,758::task::993::Storage.TaskManager.Task::(_decref) Task=`d34eeb1f-9fac-4609-9109-a201bb9e294a`::ref 0 aborting False >Thread-149::DEBUG::2014-10-23 17:15:11,759::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-149::DEBUG::2014-10-23 17:15:11,760::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,791::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-150::DEBUG::2014-10-23 17:15:11,793::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'LVMVolumeGroup.getInfo' in bridge with {u'lvmvolumegroupID': u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe'} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,793::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-150::DEBUG::2014-10-23 17:15:11,796::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0971e08-8a1b-4021-8561-2e0d22ce6c30`::moving from state init -> state preparing >Thread-150::INFO::2014-10-23 17:15:11,797::logUtils::44::dispatcher::(wrapper) Run and protect: getVGInfo(vgUUID=u'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', options=None) >Thread-150::DEBUG::2014-10-23 17:15:11,821::lvm::317::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' got the operation mutex >Thread-150::DEBUG::2014-10-23 17:15:11,822::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm pvs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,size,vg_name,vg_uuid,pe_start,pe_count,pe_alloc_count,mda_count,dev_size /dev/mapper/1cmg02 (cwd None) >Thread-150::DEBUG::2014-10-23 17:15:11,919::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-150::DEBUG::2014-10-23 17:15:11,920::lvm::342::Storage.OperationMutex::(_reloadpvs) Operation 'lvm reload operation' released the operation mutex >Thread-150::INFO::2014-10-23 17:15:11,921::logUtils::47::dispatcher::(wrapper) Run and protect: getVGInfo, Return response: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-150::DEBUG::2014-10-23 17:15:11,922::task::1191::Storage.TaskManager.Task::(prepare) Task=`f0971e08-8a1b-4021-8561-2e0d22ce6c30`::finished: {'info': {'state': 'OK', 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}}} >Thread-150::DEBUG::2014-10-23 17:15:11,923::task::595::Storage.TaskManager.Task::(_updateState) Task=`f0971e08-8a1b-4021-8561-2e0d22ce6c30`::moving from state preparing -> state finished >Thread-150::DEBUG::2014-10-23 17:15:11,923::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-150::DEBUG::2014-10-23 17:15:11,924::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-150::DEBUG::2014-10-23 17:15:11,924::task::993::Storage.TaskManager.Task::(_decref) Task=`f0971e08-8a1b-4021-8561-2e0d22ce6c30`::ref 0 aborting False >Thread-150::WARNING::2014-10-23 17:15:11,925::Bridge::212::root::(_typeFixup) Fixing up type VolumeGroupInfo >Thread-150::WARNING::2014-10-23 17:15:11,926::Bridge::467::root::(fieldClone) fieldClone: type -> deviceType >Thread-150::DEBUG::2014-10-23 17:15:11,928::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'LVMVolumeGroup.getInfo' in bridge with {'state': 'OK', 'deviceType': 3, 'vgsize': '101602820096', 'name': 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'vgfree': '97442070528', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pvlist': [{'vendorID': 'RH-str01', 'capacity': '101602820096', 'fwrev': '0000', 'vgUUID': 'dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'pathlist': [{'connection': '10.34.63.202', 'iqn': 'iqn.2012-10.brq.str-01:cmestreg', 'portal': '1', 'port': '3260', 'initiatorname': 'default'}], 'pathstatus': [{'physdev': 'sdc', 'type': 'iSCSI', 'state': 'active', 'lun': '2'}], 'devtype': 'iSCSI', 'pvUUID': 'qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr', 'serial': 'SRH-str01VIRTUAL-DISK', 'GUID': '1cmg02', 'devcapacity': '102005473280', 'productID': 'VIRTUAL-DISK'}], 'type': 3, 'attr': {'allocation': 'n', 'partial': '-', 'exported': '-', 'permission': 'w', 'clustered': '-', 'resizeable': 'z'}} >Thread-150::DEBUG::2014-10-23 17:15:11,932::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:11,941::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-151::DEBUG::2014-10-23 17:15:11,944::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.activate' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >JsonRpcServer::DEBUG::2014-10-23 17:15:11,944::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-151::DEBUG::2014-10-23 17:15:11,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::moving from state init -> state preparing >Thread-151::INFO::2014-10-23 17:15:11,947::logUtils::44::dispatcher::(wrapper) Run and protect: activateStorageDomain(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-151::DEBUG::2014-10-23 17:15:11,949::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`713d322c-c3d7-477d-be7d-53c80bd3d928`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1249' at 'activateStorageDomain' >Thread-151::DEBUG::2014-10-23 17:15:11,949::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-151::DEBUG::2014-10-23 17:15:11,950::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-151::DEBUG::2014-10-23 17:15:11,950::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`713d322c-c3d7-477d-be7d-53c80bd3d928`::Granted request >Thread-151::DEBUG::2014-10-23 17:15:11,951::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-151::DEBUG::2014-10-23 17:15:11,952::task::993::Storage.TaskManager.Task::(_decref) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::ref 1 aborting False >Thread-151::DEBUG::2014-10-23 17:15:11,953::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`729c495f-3039-493c-b7de-239f667da475`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1250' at 'activateStorageDomain' >Thread-151::DEBUG::2014-10-23 17:15:11,953::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'exclusive' >Thread-151::DEBUG::2014-10-23 17:15:11,954::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'exclusive' (1 active user) >Thread-151::DEBUG::2014-10-23 17:15:11,954::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`729c495f-3039-493c-b7de-239f667da475`::Granted request >Thread-151::DEBUG::2014-10-23 17:15:11,955::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (exclusive) >Thread-151::DEBUG::2014-10-23 17:15:11,956::task::993::Storage.TaskManager.Task::(_decref) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::ref 1 aborting False >Thread-151::INFO::2014-10-23 17:15:11,956::sp::1031::Storage.StoragePool::(activateSD) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f spUUID=00000002-0002-0002-0002-000000000067 >Thread-151::DEBUG::2014-10-23 17:15:11,956::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-151::DEBUG::2014-10-23 17:15:11,957::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-151::DEBUG::2014-10-23 17:15:11,957::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-151::DEBUG::2014-10-23 17:15:11,957::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-151::DEBUG::2014-10-23 17:15:11,958::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-151::DEBUG::2014-10-23 17:15:11,958::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:11,970::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-151::DEBUG::2014-10-23 17:15:11,972::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,053::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,053::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,053::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,054::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,054::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,054::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,054::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,054::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-151::DEBUG::2014-10-23 17:15:12,054::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,056::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,157::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,157::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,157::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,157::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,163::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000592901 s, 3.5 MB/s\n'; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,163::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000592901 s, 3.5 MB/s'], size: 2048 >Thread-151::DEBUG::2014-10-23 17:15:12,163::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-151::DEBUG::2014-10-23 17:15:12,163::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-151::DEBUG::2014-10-23 17:15:12,163::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,163::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,164::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,164::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,164::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,164::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,164::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,258::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,258::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,259::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-151::INFO::2014-10-23 17:15:12,259::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS already registered >Thread-151::INFO::2014-10-23 17:15:12,259::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS already registered >Thread-151::INFO::2014-10-23 17:15:12,259::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS already registered >Thread-151::DEBUG::2014-10-23 17:15:12,260::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images >Thread-151::WARNING::2014-10-23 17:15:12,260::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images already exists >Thread-151::DEBUG::2014-10-23 17:15:12,260::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md >Thread-151::WARNING::2014-10-23 17:15:12,260::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md already exists >Thread-151::INFO::2014-10-23 17:15:12,260::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f to /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-151::DEBUG::2014-10-23 17:15:12,260::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-151::DEBUG::2014-10-23 17:15:12,260::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,260::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,355::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,356::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,356::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,356::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,361::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000566632 s, 3.6 MB/s\n'; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,361::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000566632 s, 3.6 MB/s'], size: 2048 >Thread-151::DEBUG::2014-10-23 17:15:12,361::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-151::DEBUG::2014-10-23 17:15:12,362::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-151::DEBUG::2014-10-23 17:15:12,362::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,362::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,362::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,362::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,362::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,362::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,363::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,458::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,458::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,459::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-151::INFO::2014-10-23 17:15:12,459::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-151::INFO::2014-10-23 17:15:12,459::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-151::INFO::2014-10-23 17:15:12,459::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-151::DEBUG::2014-10-23 17:15:12,460::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-151::DEBUG::2014-10-23 17:15:12,460::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-151::DEBUG::2014-10-23 17:15:12,460::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,460::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,465::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000500683 s, 4.1 MB/s\n'; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,466::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000500683 s, 4.1 MB/s'], size: 2048 >Thread-151::DEBUG::2014-10-23 17:15:12,466::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-151::DEBUG::2014-10-23 17:15:12,466::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-151::DEBUG::2014-10-23 17:15:12,466::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-151::DEBUG::2014-10-23 17:15:12,466::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,466::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,466::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,467::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,467::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,467::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,560::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-151::DEBUG::2014-10-23 17:15:12,560::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-151::DEBUG::2014-10-23 17:15:12,560::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-151::INFO::2014-10-23 17:15:12,561::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS already registered >Thread-151::INFO::2014-10-23 17:15:12,561::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS already registered >Thread-151::INFO::2014-10-23 17:15:12,561::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS already registered >Thread-151::INFO::2014-10-23 17:15:12,561::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': 'Active'} >Thread-151::DEBUG::2014-10-23 17:15:12,561::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-151::DEBUG::2014-10-23 17:15:12,562::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-151::INFO::2014-10-23 17:15:12,562::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-151::DEBUG::2014-10-23 17:15:12,562::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` started monitoring domain `dae2b061-ea53-4ca2-8b91-f24f0641dc3f` >Thread-152::DEBUG::2014-10-23 17:15:12,562::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-151::DEBUG::2014-10-23 17:15:12,563::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-151::INFO::2014-10-23 17:15:12,563::logUtils::47::dispatcher::(wrapper) Run and protect: activateStorageDomain, Return response: None >Thread-152::DEBUG::2014-10-23 17:15:12,564::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-151::DEBUG::2014-10-23 17:15:12,564::task::1191::Storage.TaskManager.Task::(prepare) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::finished: None >Thread-151::DEBUG::2014-10-23 17:15:12,569::task::595::Storage.TaskManager.Task::(_updateState) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::moving from state preparing -> state finished >Thread-151::DEBUG::2014-10-23 17:15:12,569::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-151::DEBUG::2014-10-23 17:15:12,569::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-151::DEBUG::2014-10-23 17:15:12,570::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-151::DEBUG::2014-10-23 17:15:12,571::task::993::Storage.TaskManager.Task::(_decref) Task=`47dfb1c1-477f-4720-81b4-eca662cbcd03`::ref 0 aborting False >Thread-151::DEBUG::2014-10-23 17:15:12,571::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.activate' in bridge with True >Thread-151::DEBUG::2014-10-23 17:15:12,571::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:12,603::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:12,604::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-153::DEBUG::2014-10-23 17:15:12,604::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-153::DEBUG::2014-10-23 17:15:12,605::task::595::Storage.TaskManager.Task::(_updateState) Task=`1510b5aa-35e9-4988-8f16-6bd3e3ba9273`::moving from state init -> state preparing >Thread-153::INFO::2014-10-23 17:15:12,605::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'active'}, options=None) >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`9e92111b-c1ca-4974-9d8b-76ac84c2a6ba`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`9e92111b-c1ca-4974-9d8b-76ac84c2a6ba`::Granted request >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f1ef32d4-50cd-4ef1-a236-3b7170417f45`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-153::DEBUG::2014-10-23 17:15:12,606::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f1ef32d4-50cd-4ef1-a236-3b7170417f45`::Granted request >Thread-153::DEBUG::2014-10-23 17:15:12,607::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'active'} >Thread-153::INFO::2014-10-23 17:15:12,608::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'Active'} >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,608::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-153::DEBUG::2014-10-23 17:15:12,609::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-153::DEBUG::2014-10-23 17:15:12,609::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-153::DEBUG::2014-10-23 17:15:12,609::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-153::DEBUG::2014-10-23 17:15:12,609::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-153::DEBUG::2014-10-23 17:15:12,609::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:12,622::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-153::DEBUG::2014-10-23 17:15:12,622::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:12,659::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:12,659::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-153::DEBUG::2014-10-23 17:15:12,683::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:12,683::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,683::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,684::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,684::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,684::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,684::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,684::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-152::DEBUG::2014-10-23 17:15:12,684::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-152::DEBUG::2014-10-23 17:15:12,685::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-153::DEBUG::2014-10-23 17:15:12,685::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:12,686::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-152::DEBUG::2014-10-23 17:15:12,686::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-152::DEBUG::2014-10-23 17:15:12,688::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:12,694::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:12,700::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-152::DEBUG::2014-10-23 17:15:12,700::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:12,791::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:12,791::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,791::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-153::DEBUG::2014-10-23 17:15:12,792::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:12,797::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000498194 s, 4.1 MB/s\n'; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:12,797::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000498194 s, 4.1 MB/s'], size: 2048 >Thread-153::DEBUG::2014-10-23 17:15:12,797::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-153::DEBUG::2014-10-23 17:15:12,797::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-153::DEBUG::2014-10-23 17:15:12,797::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-153::DEBUG::2014-10-23 17:15:12,798::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,798::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,798::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,798::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,798::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,799::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:12,805::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:12,805::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Dummy-120::DEBUG::2014-10-23 17:15:12,861::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:12,905::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0121434 s, 84.3 MB/s\n'; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:12,906::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:12,907::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:12,908::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,909::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-152::DEBUG::2014-10-23 17:15:12,910::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:12,912::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:12,913::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:12,913::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:12,914::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm reload operation' is holding the operation mutex, waiting... >Thread-153::DEBUG::2014-10-23 17:15:12,918::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:13,015::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:13,016::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-153::INFO::2014-10-23 17:15:13,016::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-152::DEBUG::2014-10-23 17:15:13,016::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::INFO::2014-10-23 17:15:13,017::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-153::INFO::2014-10-23 17:15:13,017::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-152::DEBUG::2014-10-23 17:15:13,017::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:13,018::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-152::DEBUG::2014-10-23 17:15:13,018::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-153::DEBUG::2014-10-23 17:15:13,019::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-153::DEBUG::2014-10-23 17:15:13,019::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-152::DEBUG::2014-10-23 17:15:13,019::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:13,020::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-153::DEBUG::2014-10-23 17:15:13,020::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-152::DEBUG::2014-10-23 17:15:13,020::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-153::WARNING::2014-10-23 17:15:13,021::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-153::DEBUG::2014-10-23 17:15:13,026::lvm::365::Storage.OperationMutex::(_reloadvgs) Got the operational mutex >Thread-153::DEBUG::2014-10-23 17:15:13,026::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:13,119::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:13,120::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-152::DEBUG::2014-10-23 17:15:13,120::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-153::DEBUG::2014-10-23 17:15:13,125::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:13,126::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000380309 s, 5.4 MB/s\n'; <rc> = 0 >Thread-153::DEBUG::2014-10-23 17:15:13,127::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:13,127::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000380309 s, 5.4 MB/s'], size: 2048 >Thread-153::DEBUG::2014-10-23 17:15:13,128::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md >Thread-152::DEBUG::2014-10-23 17:15:13,128::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-153::WARNING::2014-10-23 17:15:13,128::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/dom_md already exists >Thread-152::DEBUG::2014-10-23 17:15:13,129::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-153::DEBUG::2014-10-23 17:15:13,129::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images >Thread-152::DEBUG::2014-10-23 17:15:13,130::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-153::WARNING::2014-10-23 17:15:13,130::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/images already exists >Thread-152::DEBUG::2014-10-23 17:15:13,131::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::INFO::2014-10-23 17:15:13,131::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/dae2b061-ea53-4ca2-8b91-f24f0641dc3f to /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-153::DEBUG::2014-10-23 17:15:13,132::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-152::DEBUG::2014-10-23 17:15:13,132::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:13,133::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-153::WARNING::2014-10-23 17:15:13,133::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-152::DEBUG::2014-10-23 17:15:13,134::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:13,134::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-153::WARNING::2014-10-23 17:15:13,134::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-152::DEBUG::2014-10-23 17:15:13,134::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-153::INFO::2014-10-23 17:15:13,135::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-152::DEBUG::2014-10-23 17:15:13,135::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-153::DEBUG::2014-10-23 17:15:13,136::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-152::DEBUG::2014-10-23 17:15:13,136::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-153::INFO::2014-10-23 17:15:13,136::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-153::DEBUG::2014-10-23 17:15:13,141::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-153::DEBUG::2014-10-23 17:15:13,142::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-153::DEBUG::2014-10-23 17:15:13,143::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-153::INFO::2014-10-23 17:15:13,143::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-153::DEBUG::2014-10-23 17:15:13,143::task::1191::Storage.TaskManager.Task::(prepare) Task=`1510b5aa-35e9-4988-8f16-6bd3e3ba9273`::finished: True >Thread-153::DEBUG::2014-10-23 17:15:13,143::task::595::Storage.TaskManager.Task::(_updateState) Task=`1510b5aa-35e9-4988-8f16-6bd3e3ba9273`::moving from state preparing -> state finished >Thread-153::DEBUG::2014-10-23 17:15:13,143::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-153::DEBUG::2014-10-23 17:15:13,143::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-153::DEBUG::2014-10-23 17:15:13,143::task::993::Storage.TaskManager.Task::(_decref) Task=`1510b5aa-35e9-4988-8f16-6bd3e3ba9273`::ref 0 aborting False >Thread-153::DEBUG::2014-10-23 17:15:13,143::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-153::DEBUG::2014-10-23 17:15:13,144::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-152::DEBUG::2014-10-23 17:15:13,229::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:13,229::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-152::DEBUG::2014-10-23 17:15:13,229::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-152::INFO::2014-10-23 17:15:13,230::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS already registered >Thread-152::INFO::2014-10-23 17:15:13,230::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS already registered >Thread-152::INFO::2014-10-23 17:15:13,230::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS already registered >Thread-152::DEBUG::2014-10-23 17:15:13,230::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:13,235::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000547263 s, 7.5 MB/s\n'; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:13,236::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f changed its status to Valid >Thread-154::DEBUG::2014-10-23 17:15:13,236::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-152::INFO::2014-10-23 17:15:13,236::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (id: 1) >Thread-154::DEBUG::2014-10-23 17:15:13,237::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-152::DEBUG::2014-10-23 17:15:13,237::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f successfully acquired (id: 1) >Thread-154::DEBUG::2014-10-23 17:15:13,238::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `_upgradePoolDomain` >Thread-154::DEBUG::2014-10-23 17:15:13,238::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-155::INFO::2014-10-23 17:15:13,243::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:13,445::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:13,447::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-157::DEBUG::2014-10-23 17:15:13,448::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:14,932::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:14,979::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0148787 s, 68.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:16,484::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:16,485::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-158::DEBUG::2014-10-23 17:15:16,486::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:16,988::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:17,029::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125589 s, 81.5 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:17,561::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:17,661::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:17,661::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:17,666::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000523761 s, 7.8 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:15:17,669::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:17,669::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:17,669::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:17,670::__init__::232::IOProcess::(_processLogs) (23) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:17,670::__init__::232::IOProcess::(_processLogs) (23) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:17,670::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:17,670::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:17,671::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:17,671::__init__::232::IOProcess::(_processLogs) (24) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:17,671::__init__::232::IOProcess::(_processLogs) (24) Queuing response >Dummy-120::DEBUG::2014-10-23 17:15:19,038::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:19,079::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122173 s, 83.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:19,517::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:19,520::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-159::DEBUG::2014-10-23 17:15:19,521::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:21,088::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:21,129::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125307 s, 81.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:21,368::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:21,369::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-160::DEBUG::2014-10-23 17:15:21,369::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-160::DEBUG::2014-10-23 17:15:21,370::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fcf6199-e417-425c-885c-5815bb39515d`::moving from state init -> state preparing >Thread-160::INFO::2014-10-23 17:15:21,370::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-160::INFO::2014-10-23 17:15:21,382::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-160::DEBUG::2014-10-23 17:15:21,382::task::1191::Storage.TaskManager.Task::(prepare) Task=`7fcf6199-e417-425c-885c-5815bb39515d`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-160::DEBUG::2014-10-23 17:15:21,382::task::595::Storage.TaskManager.Task::(_updateState) Task=`7fcf6199-e417-425c-885c-5815bb39515d`::moving from state preparing -> state finished >Thread-160::DEBUG::2014-10-23 17:15:21,382::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-160::DEBUG::2014-10-23 17:15:21,382::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-160::DEBUG::2014-10-23 17:15:21,382::task::993::Storage.TaskManager.Task::(_decref) Task=`7fcf6199-e417-425c-885c-5815bb39515d`::ref 0 aborting False >Thread-160::DEBUG::2014-10-23 17:15:21,382::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-160::DEBUG::2014-10-23 17:15:21,383::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:21,392::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-161::DEBUG::2014-10-23 17:15:21,393::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:15:21,393::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-161::DEBUG::2014-10-23 17:15:21,395::task::595::Storage.TaskManager.Task::(_updateState) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::moving from state init -> state preparing >Thread-161::INFO::2014-10-23 17:15:21,395::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-161::DEBUG::2014-10-23 17:15:21,395::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a65049fc-734a-438f-8065-ed12cc7bd10d`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-161::DEBUG::2014-10-23 17:15:21,395::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-161::DEBUG::2014-10-23 17:15:21,395::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-161::DEBUG::2014-10-23 17:15:21,395::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a65049fc-734a-438f-8065-ed12cc7bd10d`::Granted request >Thread-161::DEBUG::2014-10-23 17:15:21,396::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-161::DEBUG::2014-10-23 17:15:21,396::task::993::Storage.TaskManager.Task::(_decref) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::ref 1 aborting False >Thread-161::INFO::2014-10-23 17:15:21,407::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Active', 'diskfree': '97442070528', 'isoprefix': '', 'alerts': [], 'disktotal': '101602820096', 'version': 3}}} >Thread-161::DEBUG::2014-10-23 17:15:21,407::task::1191::Storage.TaskManager.Task::(prepare) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Active', 'diskfree': '97442070528', 'isoprefix': '', 'alerts': [], 'disktotal': '101602820096', 'version': 3}}} >Thread-161::DEBUG::2014-10-23 17:15:21,407::task::595::Storage.TaskManager.Task::(_updateState) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::moving from state preparing -> state finished >Thread-161::DEBUG::2014-10-23 17:15:21,407::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-161::DEBUG::2014-10-23 17:15:21,407::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-161::DEBUG::2014-10-23 17:15:21,407::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-161::DEBUG::2014-10-23 17:15:21,407::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-161::DEBUG::2014-10-23 17:15:21,407::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-161::DEBUG::2014-10-23 17:15:21,408::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-161::DEBUG::2014-10-23 17:15:21,408::task::993::Storage.TaskManager.Task::(_decref) Task=`644aebbd-364a-4b10-836c-e387ad8fab21`::ref 0 aborting False >Thread-161::DEBUG::2014-10-23 17:15:21,408::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Active', 'diskfree': '97442070528', 'isoprefix': '', 'alerts': [], 'disktotal': '101602820096', 'version': 3}}} >Thread-161::DEBUG::2014-10-23 17:15:21,408::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:22,543::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:22,544::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-162::DEBUG::2014-10-23 17:15:22,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`75b61293-7854-47d2-b9d0-6c64f3a77da7`::moving from state init -> state preparing >Thread-162::INFO::2014-10-23 17:15:22,545::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-162::INFO::2014-10-23 17:15:22,545::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000523761', 'lastCheck': '4.9', 'valid': True}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000547263', 'lastCheck': '9.3', 'valid': True}} >Thread-162::DEBUG::2014-10-23 17:15:22,545::task::1191::Storage.TaskManager.Task::(prepare) Task=`75b61293-7854-47d2-b9d0-6c64f3a77da7`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000523761', 'lastCheck': '4.9', 'valid': True}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000547263', 'lastCheck': '9.3', 'valid': True}} >Thread-162::DEBUG::2014-10-23 17:15:22,545::task::595::Storage.TaskManager.Task::(_updateState) Task=`75b61293-7854-47d2-b9d0-6c64f3a77da7`::moving from state preparing -> state finished >Thread-162::DEBUG::2014-10-23 17:15:22,545::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-162::DEBUG::2014-10-23 17:15:22,546::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-162::DEBUG::2014-10-23 17:15:22,546::task::993::Storage.TaskManager.Task::(_decref) Task=`75b61293-7854-47d2-b9d0-6c64f3a77da7`::ref 0 aborting False >Thread-162::DEBUG::2014-10-23 17:15:22,547::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:22,619::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:22,620::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-163::DEBUG::2014-10-23 17:15:22,621::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:23,137::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:23,182::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0129657 s, 79.0 MB/s\n'; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:23,238::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:23,335::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:23,335::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-152::DEBUG::2014-10-23 17:15:23,341::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000573075 s, 7.1 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:15:25,192::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:25,233::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125036 s, 81.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:25,327::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:25,328::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-164::DEBUG::2014-10-23 17:15:25,328::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.deactivate' in bridge with {u'masterVersion': 1, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'00000000-0000-0000-0000-000000000000', u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-164::DEBUG::2014-10-23 17:15:25,329::task::595::Storage.TaskManager.Task::(_updateState) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::moving from state init -> state preparing >Thread-164::INFO::2014-10-23 17:15:25,329::logUtils::44::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', spUUID=u'00000002-0002-0002-0002-000000000067', msdUUID=u'00000000-0000-0000-0000-000000000000', masterVersion=1, options=None) >Thread-164::DEBUG::2014-10-23 17:15:25,329::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f800aa0c-86ce-4576-9d83-4e1c7a9093e7`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1227' at 'deactivateStorageDomain' >Thread-164::DEBUG::2014-10-23 17:15:25,330::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-164::DEBUG::2014-10-23 17:15:25,330::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-164::DEBUG::2014-10-23 17:15:25,330::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f800aa0c-86ce-4576-9d83-4e1c7a9093e7`::Granted request >Thread-164::DEBUG::2014-10-23 17:15:25,330::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-164::DEBUG::2014-10-23 17:15:25,330::task::993::Storage.TaskManager.Task::(_decref) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::ref 1 aborting False >Thread-164::DEBUG::2014-10-23 17:15:25,330::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`8b8d6b98-5ffe-490c-9a59-3f1d96ac989c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1228' at 'deactivateStorageDomain' >Thread-164::DEBUG::2014-10-23 17:15:25,330::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'exclusive' >Thread-164::DEBUG::2014-10-23 17:15:25,331::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'exclusive' (1 active user) >Thread-164::DEBUG::2014-10-23 17:15:25,331::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`8b8d6b98-5ffe-490c-9a59-3f1d96ac989c`::Granted request >Thread-164::DEBUG::2014-10-23 17:15:25,331::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (exclusive) >Thread-164::DEBUG::2014-10-23 17:15:25,331::task::993::Storage.TaskManager.Task::(_decref) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::ref 1 aborting False >Thread-164::INFO::2014-10-23 17:15:25,331::sp::1070::Storage.StoragePool::(deactivateSD) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f spUUID=00000002-0002-0002-0002-000000000067 newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-164::INFO::2014-10-23 17:15:25,332::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': 'Attached'} >Thread-164::DEBUG::2014-10-23 17:15:25,332::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-164::DEBUG::2014-10-23 17:15:25,332::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-164::INFO::2014-10-23 17:15:25,332::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-152::DEBUG::2014-10-23 17:15:25,332::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-152::INFO::2014-10-23 17:15:25,333::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:25,816::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:25,817::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-165::DEBUG::2014-10-23 17:15:25,818::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:27,242::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:27,283::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0127702 s, 80.2 MB/s\n'; <rc> = 0 >Thread-152::DEBUG::2014-10-23 17:15:27,333::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f released successfully (id: 1) >Thread-164::DEBUG::2014-10-23 17:15:27,333::sp::1407::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` stopped monitoring domain `dae2b061-ea53-4ca2-8b91-f24f0641dc3f` >Thread-164::DEBUG::2014-10-23 17:15:27,333::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-164::INFO::2014-10-23 17:15:27,334::logUtils::47::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-164::DEBUG::2014-10-23 17:15:27,334::task::1191::Storage.TaskManager.Task::(prepare) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::finished: None >Thread-164::DEBUG::2014-10-23 17:15:27,334::task::595::Storage.TaskManager.Task::(_updateState) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::moving from state preparing -> state finished >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-164::DEBUG::2014-10-23 17:15:27,334::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-164::DEBUG::2014-10-23 17:15:27,335::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-164::DEBUG::2014-10-23 17:15:27,335::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-164::DEBUG::2014-10-23 17:15:27,335::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-164::DEBUG::2014-10-23 17:15:27,335::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-164::DEBUG::2014-10-23 17:15:27,335::task::993::Storage.TaskManager.Task::(_decref) Task=`03efb213-a34c-421f-9743-0dc553edd79f`::ref 0 aborting False >Thread-164::DEBUG::2014-10-23 17:15:27,335::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.deactivate' in bridge with True >Thread-164::DEBUG::2014-10-23 17:15:27,335::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:27,362::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:27,363::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-166::DEBUG::2014-10-23 17:15:27,363::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'attached'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-166::DEBUG::2014-10-23 17:15:27,364::task::595::Storage.TaskManager.Task::(_updateState) Task=`442dac63-7405-4b04-b57b-5fe4b1c185fb`::moving from state init -> state preparing >Thread-166::INFO::2014-10-23 17:15:27,364::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'attached'}, options=None) >Thread-166::DEBUG::2014-10-23 17:15:27,364::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`8235e823-73d5-4ec5-88be-82fb080d8285`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-166::DEBUG::2014-10-23 17:15:27,364::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-166::DEBUG::2014-10-23 17:15:27,364::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-166::DEBUG::2014-10-23 17:15:27,365::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`8235e823-73d5-4ec5-88be-82fb080d8285`::Granted request >Thread-166::DEBUG::2014-10-23 17:15:27,365::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f06bb9ed-643b-4126-b268-fbced7b630cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-166::DEBUG::2014-10-23 17:15:27,365::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-166::DEBUG::2014-10-23 17:15:27,365::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-166::DEBUG::2014-10-23 17:15:27,365::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`f06bb9ed-643b-4126-b268-fbced7b630cb`::Granted request >Thread-166::DEBUG::2014-10-23 17:15:27,365::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'attached'} >Thread-166::INFO::2014-10-23 17:15:27,365::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active', u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': u'Attached'} >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,366::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-166::DEBUG::2014-10-23 17:15:27,366::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-166::DEBUG::2014-10-23 17:15:27,366::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-166::DEBUG::2014-10-23 17:15:27,367::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-166::DEBUG::2014-10-23 17:15:27,367::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-166::DEBUG::2014-10-23 17:15:27,367::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,380::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-166::DEBUG::2014-10-23 17:15:27,380::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,441::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,442::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,443::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-166::DEBUG::2014-10-23 17:15:27,443::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,444::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,545::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-166::DEBUG::2014-10-23 17:15:27,545::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,546::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-166::DEBUG::2014-10-23 17:15:27,546::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,551::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000536692 s, 3.8 MB/s\n'; <rc> = 0 >Thread-166::DEBUG::2014-10-23 17:15:27,551::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000536692 s, 3.8 MB/s'], size: 2048 >Thread-166::DEBUG::2014-10-23 17:15:27,552::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-166::DEBUG::2014-10-23 17:15:27,552::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-166::DEBUG::2014-10-23 17:15:27,552::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-166::DEBUG::2014-10-23 17:15:27,552::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,552::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,552::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,552::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,552::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,553::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,649::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-166::DEBUG::2014-10-23 17:15:27,649::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,649::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-166::INFO::2014-10-23 17:15:27,650::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-166::INFO::2014-10-23 17:15:27,650::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-166::INFO::2014-10-23 17:15:27,650::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-166::DEBUG::2014-10-23 17:15:27,650::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-166::DEBUG::2014-10-23 17:15:27,650::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-166::DEBUG::2014-10-23 17:15:27,650::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-166::DEBUG::2014-10-23 17:15:27,650::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-166::DEBUG::2014-10-23 17:15:27,651::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-166::WARNING::2014-10-23 17:15:27,651::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-166::DEBUG::2014-10-23 17:15:27,651::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,651::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:27,672::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-166::DEBUG::2014-10-23 17:15:27,758::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-166::DEBUG::2014-10-23 17:15:27,758::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-166::DEBUG::2014-10-23 17:15:27,759::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-166::WARNING::2014-10-23 17:15:27,759::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-166::DEBUG::2014-10-23 17:15:27,759::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-166::WARNING::2014-10-23 17:15:27,759::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-166::INFO::2014-10-23 17:15:27,759::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-166::DEBUG::2014-10-23 17:15:27,759::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-166::INFO::2014-10-23 17:15:27,759::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-166::DEBUG::2014-10-23 17:15:27,759::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-166::DEBUG::2014-10-23 17:15:27,760::sp::1221::Storage.StoragePool::(__rebuild) collecting stale storage domain link /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-166::DEBUG::2014-10-23 17:15:27,760::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-166::DEBUG::2014-10-23 17:15:27,761::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-166::INFO::2014-10-23 17:15:27,761::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-166::DEBUG::2014-10-23 17:15:27,761::task::1191::Storage.TaskManager.Task::(prepare) Task=`442dac63-7405-4b04-b57b-5fe4b1c185fb`::finished: True >Thread-166::DEBUG::2014-10-23 17:15:27,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`442dac63-7405-4b04-b57b-5fe4b1c185fb`::moving from state preparing -> state finished >Thread-166::DEBUG::2014-10-23 17:15:27,761::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-166::DEBUG::2014-10-23 17:15:27,761::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-166::DEBUG::2014-10-23 17:15:27,761::task::993::Storage.TaskManager.Task::(_decref) Task=`442dac63-7405-4b04-b57b-5fe4b1c185fb`::ref 0 aborting False >Thread-166::DEBUG::2014-10-23 17:15:27,761::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-166::DEBUG::2014-10-23 17:15:27,761::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:15:27,766::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:27,766::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:27,771::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00058642 s, 7.0 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:15:27,772::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:27,772::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:27,773::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:27,773::__init__::232::IOProcess::(_processLogs) (25) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:27,773::__init__::232::IOProcess::(_processLogs) (25) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:27,774::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:27,774::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:27,774::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:27,775::__init__::232::IOProcess::(_processLogs) (26) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:27,775::__init__::232::IOProcess::(_processLogs) (26) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:27,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:27,864::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-167::DEBUG::2014-10-23 17:15:27,864::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnectStorageServer' in bridge with {u'connectionParams': [], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-167::DEBUG::2014-10-23 17:15:27,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee7bfe2c-0ec5-41bd-9281-8d967b454a95`::moving from state init -> state preparing >Thread-167::INFO::2014-10-23 17:15:27,865::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[], options=None) >Thread-167::DEBUG::2014-10-23 17:15:27,865::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-167::DEBUG::2014-10-23 17:15:27,865::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-167::DEBUG::2014-10-23 17:15:27,865::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-167::DEBUG::2014-10-23 17:15:27,866::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-167::DEBUG::2014-10-23 17:15:27,866::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-167::DEBUG::2014-10-23 17:15:27,866::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-167::DEBUG::2014-10-23 17:15:27,878::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-167::DEBUG::2014-10-23 17:15:27,878::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-167::DEBUG::2014-10-23 17:15:27,957::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-167::DEBUG::2014-10-23 17:15:27,958::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-167::INFO::2014-10-23 17:15:27,959::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': []} >Thread-167::DEBUG::2014-10-23 17:15:27,959::task::1191::Storage.TaskManager.Task::(prepare) Task=`ee7bfe2c-0ec5-41bd-9281-8d967b454a95`::finished: {'statuslist': []} >Thread-167::DEBUG::2014-10-23 17:15:27,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`ee7bfe2c-0ec5-41bd-9281-8d967b454a95`::moving from state preparing -> state finished >Thread-167::DEBUG::2014-10-23 17:15:27,959::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-167::DEBUG::2014-10-23 17:15:27,959::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-167::DEBUG::2014-10-23 17:15:27,959::task::993::Storage.TaskManager.Task::(_decref) Task=`ee7bfe2c-0ec5-41bd-9281-8d967b454a95`::ref 0 aborting False >Thread-167::DEBUG::2014-10-23 17:15:27,959::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnectStorageServer' in bridge with [] >Thread-167::DEBUG::2014-10-23 17:15:27,959::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:28,833::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:28,834::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-168::DEBUG::2014-10-23 17:15:28,835::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:29,292::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:29,333::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0128989 s, 79.4 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:15:31,342::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:31,383::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0126152 s, 81.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:31,504::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:31,506::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-169::DEBUG::2014-10-23 17:15:31,506::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-169::DEBUG::2014-10-23 17:15:31,507::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1895e3f-5627-47e1-95d4-dd5a13edecb5`::moving from state init -> state preparing >Thread-169::INFO::2014-10-23 17:15:31,507::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-169::INFO::2014-10-23 17:15:31,518::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-169::DEBUG::2014-10-23 17:15:31,518::task::1191::Storage.TaskManager.Task::(prepare) Task=`c1895e3f-5627-47e1-95d4-dd5a13edecb5`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-169::DEBUG::2014-10-23 17:15:31,518::task::595::Storage.TaskManager.Task::(_updateState) Task=`c1895e3f-5627-47e1-95d4-dd5a13edecb5`::moving from state preparing -> state finished >Thread-169::DEBUG::2014-10-23 17:15:31,518::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-169::DEBUG::2014-10-23 17:15:31,518::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-169::DEBUG::2014-10-23 17:15:31,518::task::993::Storage.TaskManager.Task::(_decref) Task=`c1895e3f-5627-47e1-95d4-dd5a13edecb5`::ref 0 aborting False >Thread-169::DEBUG::2014-10-23 17:15:31,518::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-169::DEBUG::2014-10-23 17:15:31,519::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:31,528::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:31,529::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-170::DEBUG::2014-10-23 17:15:31,529::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-170::DEBUG::2014-10-23 17:15:31,529::task::595::Storage.TaskManager.Task::(_updateState) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::moving from state init -> state preparing >Thread-170::INFO::2014-10-23 17:15:31,530::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-170::DEBUG::2014-10-23 17:15:31,530::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`78aa15f8-20ab-46e0-8fc8-87d0a2c44028`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-170::DEBUG::2014-10-23 17:15:31,530::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-170::DEBUG::2014-10-23 17:15:31,530::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-170::DEBUG::2014-10-23 17:15:31,530::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`78aa15f8-20ab-46e0-8fc8-87d0a2c44028`::Granted request >Thread-170::DEBUG::2014-10-23 17:15:31,531::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-170::DEBUG::2014-10-23 17:15:31,531::task::993::Storage.TaskManager.Task::(_decref) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::ref 1 aborting False >Thread-170::DEBUG::2014-10-23 17:15:31,531::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-170::DEBUG::2014-10-23 17:15:31,532::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-170::DEBUG::2014-10-23 17:15:31,627::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-170::DEBUG::2014-10-23 17:15:31,627::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-170::INFO::2014-10-23 17:15:31,639::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-170::DEBUG::2014-10-23 17:15:31,639::task::1191::Storage.TaskManager.Task::(prepare) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-170::DEBUG::2014-10-23 17:15:31,639::task::595::Storage.TaskManager.Task::(_updateState) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::moving from state preparing -> state finished >Thread-170::DEBUG::2014-10-23 17:15:31,639::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-170::DEBUG::2014-10-23 17:15:31,639::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-170::DEBUG::2014-10-23 17:15:31,639::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-170::DEBUG::2014-10-23 17:15:31,639::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-170::DEBUG::2014-10-23 17:15:31,640::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-170::DEBUG::2014-10-23 17:15:31,640::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-170::DEBUG::2014-10-23 17:15:31,640::task::993::Storage.TaskManager.Task::(_decref) Task=`79ee6755-1314-44b5-97a3-ed93d3b102cb`::ref 0 aborting False >Thread-170::DEBUG::2014-10-23 17:15:31,640::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-170::DEBUG::2014-10-23 17:15:31,640::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:31,910::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:31,910::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-171::DEBUG::2014-10-23 17:15:31,911::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:33,392::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:33,436::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012395 s, 82.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:34,935::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:34,936::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-172::DEBUG::2014-10-23 17:15:34,937::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:35,446::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:35,487::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125133 s, 81.8 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:15:37,496::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:37,537::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124913 s, 82.0 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:37,775::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:37,780::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000501182 s, 8.2 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:15:37,782::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:37,782::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:37,782::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:37,782::__init__::232::IOProcess::(_processLogs) (27) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:37,783::__init__::232::IOProcess::(_processLogs) (27) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:37,783::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:37,783::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:37,784::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:37,784::__init__::232::IOProcess::(_processLogs) (28) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:37,784::__init__::232::IOProcess::(_processLogs) (28) Queuing response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:37,952::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:37,953::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-173::DEBUG::2014-10-23 17:15:37,954::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d8a2632-d95d-4ce9-9d9c-541b41169ae3`::moving from state init -> state preparing >Thread-173::INFO::2014-10-23 17:15:37,954::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-173::INFO::2014-10-23 17:15:37,954::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000501182', 'lastCheck': '0.2', 'valid': True}} >Thread-173::DEBUG::2014-10-23 17:15:37,955::task::1191::Storage.TaskManager.Task::(prepare) Task=`8d8a2632-d95d-4ce9-9d9c-541b41169ae3`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000501182', 'lastCheck': '0.2', 'valid': True}} >Thread-173::DEBUG::2014-10-23 17:15:37,955::task::595::Storage.TaskManager.Task::(_updateState) Task=`8d8a2632-d95d-4ce9-9d9c-541b41169ae3`::moving from state preparing -> state finished >Thread-173::DEBUG::2014-10-23 17:15:37,955::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-173::DEBUG::2014-10-23 17:15:37,955::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-173::DEBUG::2014-10-23 17:15:37,955::task::993::Storage.TaskManager.Task::(_decref) Task=`8d8a2632-d95d-4ce9-9d9c-541b41169ae3`::ref 0 aborting False >Thread-173::DEBUG::2014-10-23 17:15:37,957::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:37,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:37,995::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-174::DEBUG::2014-10-23 17:15:37,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:39,544::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:39,585::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0125101 s, 81.9 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:41,133::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:41,135::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-175::DEBUG::2014-10-23 17:15:41,138::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:41,595::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:41,648::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0219884 s, 46.6 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:41,752::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:41,755::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-176::DEBUG::2014-10-23 17:15:41,755::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-176::DEBUG::2014-10-23 17:15:41,761::task::595::Storage.TaskManager.Task::(_updateState) Task=`165e162e-a19f-47bc-baf1-059e73d54008`::moving from state init -> state preparing >Thread-176::INFO::2014-10-23 17:15:41,763::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-176::INFO::2014-10-23 17:15:41,775::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-176::DEBUG::2014-10-23 17:15:41,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`165e162e-a19f-47bc-baf1-059e73d54008`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-176::DEBUG::2014-10-23 17:15:41,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`165e162e-a19f-47bc-baf1-059e73d54008`::moving from state preparing -> state finished >Thread-176::DEBUG::2014-10-23 17:15:41,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-176::DEBUG::2014-10-23 17:15:41,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-176::DEBUG::2014-10-23 17:15:41,777::task::993::Storage.TaskManager.Task::(_decref) Task=`165e162e-a19f-47bc-baf1-059e73d54008`::ref 0 aborting False >Thread-176::DEBUG::2014-10-23 17:15:41,778::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-176::DEBUG::2014-10-23 17:15:41,779::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:41,795::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-177::DEBUG::2014-10-23 17:15:41,797::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:15:41,797::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-177::DEBUG::2014-10-23 17:15:41,800::task::595::Storage.TaskManager.Task::(_updateState) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::moving from state init -> state preparing >Thread-177::INFO::2014-10-23 17:15:41,801::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-177::DEBUG::2014-10-23 17:15:41,802::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a436796d-6c39-45a1-98c5-3830d85afeee`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-177::DEBUG::2014-10-23 17:15:41,803::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-177::DEBUG::2014-10-23 17:15:41,803::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-177::DEBUG::2014-10-23 17:15:41,804::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a436796d-6c39-45a1-98c5-3830d85afeee`::Granted request >Thread-177::DEBUG::2014-10-23 17:15:41,805::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-177::DEBUG::2014-10-23 17:15:41,806::task::993::Storage.TaskManager.Task::(_decref) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::ref 1 aborting False >Thread-177::INFO::2014-10-23 17:15:41,818::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-177::DEBUG::2014-10-23 17:15:41,819::task::1191::Storage.TaskManager.Task::(prepare) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-177::DEBUG::2014-10-23 17:15:41,819::task::595::Storage.TaskManager.Task::(_updateState) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::moving from state preparing -> state finished >Thread-177::DEBUG::2014-10-23 17:15:41,820::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-177::DEBUG::2014-10-23 17:15:41,820::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-177::DEBUG::2014-10-23 17:15:41,821::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-177::DEBUG::2014-10-23 17:15:41,822::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-177::DEBUG::2014-10-23 17:15:41,823::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-177::DEBUG::2014-10-23 17:15:41,823::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-177::DEBUG::2014-10-23 17:15:41,824::task::993::Storage.TaskManager.Task::(_decref) Task=`256160d5-9637-4b0f-8660-1192fa002e4b`::ref 0 aborting False >Thread-177::DEBUG::2014-10-23 17:15:41,825::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active,dae2b061-ea53-4ca2-8b91-f24f0641dc3f:Attached', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}, u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f': {'status': u'Attached', 'isoprefix': '', 'alerts': []}}} >Thread-177::DEBUG::2014-10-23 17:15:41,828::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:43,775::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:43,815::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012414 s, 82.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:44,155::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:44,156::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-178::DEBUG::2014-10-23 17:15:44,157::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:45,648::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:45,649::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-179::DEBUG::2014-10-23 17:15:45,649::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-179::DEBUG::2014-10-23 17:15:45,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`516d685d-0b64-4624-a532-0d4d2fa032cf`::moving from state init -> state preparing >Thread-179::INFO::2014-10-23 17:15:45,650::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-179::DEBUG::2014-10-23 17:15:45,650::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-179::DEBUG::2014-10-23 17:15:45,662::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-179::DEBUG::2014-10-23 17:15:45,663::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-179::DEBUG::2014-10-23 17:15:45,673::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-179::DEBUG::2014-10-23 17:15:45,673::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-179::DEBUG::2014-10-23 17:15:45,682::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-179::DEBUG::2014-10-23 17:15:45,683::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-179::DEBUG::2014-10-23 17:15:45,683::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-179::DEBUG::2014-10-23 17:15:45,791::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-179::DEBUG::2014-10-23 17:15:45,792::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-179::DEBUG::2014-10-23 17:15:45,792::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f') >Thread-179::DEBUG::2014-10-23 17:15:45,792::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain, dae2b061-ea53-4ca2-8b91-f24f0641dc3f: storage.blockSD.findDomain} >Thread-179::INFO::2014-10-23 17:15:45,792::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-179::DEBUG::2014-10-23 17:15:45,792::task::1191::Storage.TaskManager.Task::(prepare) Task=`516d685d-0b64-4624-a532-0d4d2fa032cf`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-179::DEBUG::2014-10-23 17:15:45,792::task::595::Storage.TaskManager.Task::(_updateState) Task=`516d685d-0b64-4624-a532-0d4d2fa032cf`::moving from state preparing -> state finished >Thread-179::DEBUG::2014-10-23 17:15:45,793::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-179::DEBUG::2014-10-23 17:15:45,793::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-179::DEBUG::2014-10-23 17:15:45,793::task::993::Storage.TaskManager.Task::(_decref) Task=`516d685d-0b64-4624-a532-0d4d2fa032cf`::ref 0 aborting False >Thread-179::DEBUG::2014-10-23 17:15:45,793::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-179::DEBUG::2014-10-23 17:15:45,793::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:45,813::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:45,814::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-180::DEBUG::2014-10-23 17:15:45,814::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.detach' in bridge with {u'masterVersion': 1, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'00000000-0000-0000-0000-000000000000', u'storagedomainID': u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f'} >Thread-180::DEBUG::2014-10-23 17:15:45,815::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::moving from state init -> state preparing >Thread-180::INFO::2014-10-23 17:15:45,815::logUtils::44::dispatcher::(wrapper) Run and protect: detachStorageDomain(sdUUID=u'dae2b061-ea53-4ca2-8b91-f24f0641dc3f', spUUID=u'00000002-0002-0002-0002-000000000067', msdUUID=u'00000000-0000-0000-0000-000000000000', masterVersion=1, options=None) >Thread-180::DEBUG::2014-10-23 17:15:45,815::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`d4932097-62d2-4a93-b82d-07a082ee4789`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '814' at 'detachStorageDomain' >Thread-180::DEBUG::2014-10-23 17:15:45,815::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-180::DEBUG::2014-10-23 17:15:45,816::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-180::DEBUG::2014-10-23 17:15:45,816::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`d4932097-62d2-4a93-b82d-07a082ee4789`::Granted request >Thread-180::DEBUG::2014-10-23 17:15:45,816::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-180::DEBUG::2014-10-23 17:15:45,816::task::993::Storage.TaskManager.Task::(_decref) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::ref 1 aborting False >Thread-180::DEBUG::2014-10-23 17:15:45,816::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`bd6bbe98-e80a-4ad3-94b2-f63eb3487e2b`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '815' at 'detachStorageDomain' >Thread-180::DEBUG::2014-10-23 17:15:45,816::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' for lock type 'exclusive' >Thread-180::DEBUG::2014-10-23 17:15:45,816::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free. Now locking as 'exclusive' (1 active user) >Thread-180::DEBUG::2014-10-23 17:15:45,817::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f`ReqID=`bd6bbe98-e80a-4ad3-94b2-f63eb3487e2b`::Granted request >Thread-180::DEBUG::2014-10-23 17:15:45,817::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::_resourcesAcquired: Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f (exclusive) >Thread-180::DEBUG::2014-10-23 17:15:45,817::task::993::Storage.TaskManager.Task::(_decref) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::ref 1 aborting False >Thread-180::INFO::2014-10-23 17:15:45,817::sp::954::Storage.StoragePool::(detachSD) sdUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f spUUID=00000002-0002-0002-0002-000000000067 >Thread-180::DEBUG::2014-10-23 17:15:45,817::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-180::DEBUG::2014-10-23 17:15:45,817::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-180::DEBUG::2014-10-23 17:15:45,817::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-180::DEBUG::2014-10-23 17:15:45,817::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-180::DEBUG::2014-10-23 17:15:45,817::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-180::DEBUG::2014-10-23 17:15:45,817::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:45,824::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:45,831::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-180::DEBUG::2014-10-23 17:15:45,831::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:45,866::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0122595 s, 83.5 MB/s\n'; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:45,922::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:45,922::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,922::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,923::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,923::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,923::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,923::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,923::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-180::DEBUG::2014-10-23 17:15:45,923::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:45,924::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,019::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,020::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,020::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-180::DEBUG::2014-10-23 17:15:46,020::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/dae2b061-ea53-4ca2-8b91-f24f0641dc3f/metadata count=1 (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,025::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000514705 s, 4.0 MB/s\n'; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,026::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000514705 s, 4.0 MB/s'], size: 2048 >Thread-180::DEBUG::2014-10-23 17:15:46,026::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-180::DEBUG::2014-10-23 17:15:46,026::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-180::DEBUG::2014-10-23 17:15:46,026::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-180::DEBUG::2014-10-23 17:15:46,026::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,026::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,027::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,027::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,027::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,027::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,123::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,123::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,123::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-180::INFO::2014-10-23 17:15:46,124::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_imageNS already registered >Thread-180::INFO::2014-10-23 17:15:46,124::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_volumeNS already registered >Thread-180::INFO::2014-10-23 17:15:46,124::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace dae2b061-ea53-4ca2-8b91-f24f0641dc3f_lvmActivationNS already registered >Thread-180::DEBUG::2014-10-23 17:15:46,124::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,124::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,222::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,222::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,223::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-180::DEBUG::2014-10-23 17:15:46,223::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,228::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000553248 s, 3.7 MB/s\n'; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,228::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000553248 s, 3.7 MB/s'], size: 2048 >Thread-180::DEBUG::2014-10-23 17:15:46,229::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-180::DEBUG::2014-10-23 17:15:46,229::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-180::DEBUG::2014-10-23 17:15:46,229::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-180::DEBUG::2014-10-23 17:15:46,229::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,229::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,229::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,229::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,230::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,230::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,324::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,325::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,325::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-180::INFO::2014-10-23 17:15:46,326::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-180::INFO::2014-10-23 17:15:46,326::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-180::INFO::2014-10-23 17:15:46,326::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-180::INFO::2014-10-23 17:15:46,326::sd::533::Storage.StorageDomain::(detach) detaching storage domain dae2b061-ea53-4ca2-8b91-f24f0641dc3f from pool 00000002-0002-0002-0002-000000000067 >Thread-180::DEBUG::2014-10-23 17:15:46,326::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,326::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,326::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,326::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,327::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,327::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,427::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,427::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,427::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', 'VERSION=3', 'TYPE=ISCSI', 'LOGBLKSIZE=512', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'LOCKPOLICY=', 'ROLE=Regular', 'PHYBLKSIZE=512', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'CLASS=Data', 'DESCRIPTION=iscsi_1', '_SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-180::DEBUG::2014-10-23 17:15:46,428::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-180::DEBUG::2014-10-23 17:15:46,428::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-180::DEBUG::2014-10-23 17:15:46,428::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-180::DEBUG::2014-10-23 17:15:46,428::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-180::DEBUG::2014-10-23 17:15:46,428::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi_1', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'PHYBLKSIZE=512', 'POOL_UUID=', u'PV0=pv:1cmg02,uuid:qW6hW2-WwjH-VEmd-iWb7-fuA1-pAvh-b5zkdr,pestart:0,pecount:757,mapoffset:0', 'ROLE=Regular', 'SDUUID=dae2b061-ea53-4ca2-8b91-f24f0641dc3f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=dNFeyg-Afr9-1yzJ-LSrA-UIIX-lmsl-7xjsLe', '_SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475'] >Thread-180::DEBUG::2014-10-23 17:15:46,428::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,428::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,428::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,429::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,429::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,429::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,523::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,524::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,524::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475, MDT_POOL_UUID= removing=MDT__SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca, MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 >Thread-180::DEBUG::2014-10-23 17:15:46,524::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT__SHA_CKSUM=9058c878eb0aaf97bb37b21208e3b8d0d3ea66ca --deltag MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 --addtag MDT__SHA_CKSUM=1f126101b6706302ed762875154b5add8f1e7475 --addtag MDT_POOL_UUID= dae2b061-ea53-4ca2-8b91-f24f0641dc3f (cwd None) >Thread-180::DEBUG::2014-10-23 17:15:46,697::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-180::DEBUG::2014-10-23 17:15:46,697::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,698::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-180::DEBUG::2014-10-23 17:15:46,698::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-180::WARNING::2014-10-23 17:15:46,698::sp::927::Storage.StoragePool::(forcedDetachSD) Force detaching domain `dae2b061-ea53-4ca2-8b91-f24f0641dc3f` >Thread-180::INFO::2014-10-23 17:15:46,698::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active'} >Thread-180::DEBUG::2014-10-23 17:15:46,698::sp::1143::Storage.StoragePool::(_cleanupDomainLinks) Removing: /rhev/data-center/00000002-0002-0002-0002-000000000067/dae2b061-ea53-4ca2-8b91-f24f0641dc3f >Thread-180::DEBUG::2014-10-23 17:15:46,698::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-180::DEBUG::2014-10-23 17:15:46,699::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-180::DEBUG::2014-10-23 17:15:46,699::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-180::DEBUG::2014-10-23 17:15:46,699::sp::944::Storage.StoragePool::(forcedDetachSD) Force detach for domain `dae2b061-ea53-4ca2-8b91-f24f0641dc3f` is done >Thread-180::INFO::2014-10-23 17:15:46,699::logUtils::47::dispatcher::(wrapper) Run and protect: detachStorageDomain, Return response: None >Thread-180::DEBUG::2014-10-23 17:15:46,699::task::1191::Storage.TaskManager.Task::(prepare) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::finished: None >Thread-180::DEBUG::2014-10-23 17:15:46,699::task::595::Storage.TaskManager.Task::(_updateState) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::moving from state preparing -> state finished >Thread-180::DEBUG::2014-10-23 17:15:46,699::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f': < ResourceRef 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-180::DEBUG::2014-10-23 17:15:46,699::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-180::DEBUG::2014-10-23 17:15:46,699::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' (0 active users) >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f' is free, finding out if anyone is waiting for it. >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.dae2b061-ea53-4ca2-8b91-f24f0641dc3f', Clearing records. >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-180::DEBUG::2014-10-23 17:15:46,700::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-180::DEBUG::2014-10-23 17:15:46,700::task::993::Storage.TaskManager.Task::(_decref) Task=`d5e0d271-da36-46e9-9026-0779f326a27d`::ref 0 aborting False >Thread-180::DEBUG::2014-10-23 17:15:46,701::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.detach' in bridge with True >Thread-180::DEBUG::2014-10-23 17:15:46,701::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:46,716::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:46,717::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-181::DEBUG::2014-10-23 17:15:46,717::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-181::DEBUG::2014-10-23 17:15:46,719::task::595::Storage.TaskManager.Task::(_updateState) Task=`87875458-65ff-4166-9f79-b6328fb16b53`::moving from state init -> state preparing >Thread-181::INFO::2014-10-23 17:15:46,719::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, options=None) >Thread-181::DEBUG::2014-10-23 17:15:46,719::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`090589fe-e21d-4527-9a05-7fd899eca3cb`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-181::DEBUG::2014-10-23 17:15:46,719::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-181::DEBUG::2014-10-23 17:15:46,719::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-181::DEBUG::2014-10-23 17:15:46,719::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`090589fe-e21d-4527-9a05-7fd899eca3cb`::Granted request >Thread-181::DEBUG::2014-10-23 17:15:46,720::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`55077c60-e4c8-486d-a07f-0f70ff0b6e88`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1049' at '_connectStoragePool' >Thread-181::DEBUG::2014-10-23 17:15:46,720::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-181::DEBUG::2014-10-23 17:15:46,720::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-181::DEBUG::2014-10-23 17:15:46,720::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`55077c60-e4c8-486d-a07f-0f70ff0b6e88`::Granted request >Thread-181::DEBUG::2014-10-23 17:15:46,720::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'} >Thread-181::INFO::2014-10-23 17:15:46,720::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active'} >Thread-181::DEBUG::2014-10-23 17:15:46,720::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,720::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,721::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,721::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,721::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,721::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,721::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-181::DEBUG::2014-10-23 17:15:46,721::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-181::DEBUG::2014-10-23 17:15:46,721::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-181::DEBUG::2014-10-23 17:15:46,721::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-181::DEBUG::2014-10-23 17:15:46,721::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-181::DEBUG::2014-10-23 17:15:46,721::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:46,734::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-181::DEBUG::2014-10-23 17:15:46,735::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:46,807::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-181::DEBUG::2014-10-23 17:15:46,807::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,807::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,807::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,808::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,808::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,808::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,808::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-181::DEBUG::2014-10-23 17:15:46,808::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,809::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:46,905::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-181::DEBUG::2014-10-23 17:15:46,905::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,905::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-181::DEBUG::2014-10-23 17:15:46,905::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:46,911::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000565603 s, 3.6 MB/s\n'; <rc> = 0 >Thread-181::DEBUG::2014-10-23 17:15:46,911::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000565603 s, 3.6 MB/s'], size: 2048 >Thread-181::DEBUG::2014-10-23 17:15:46,911::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-181::DEBUG::2014-10-23 17:15:46,911::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-181::DEBUG::2014-10-23 17:15:46,911::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:46,912::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:47,007::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-181::DEBUG::2014-10-23 17:15:47,007::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:47,008::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-181::INFO::2014-10-23 17:15:47,008::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-181::INFO::2014-10-23 17:15:47,008::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-181::INFO::2014-10-23 17:15:47,008::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-181::DEBUG::2014-10-23 17:15:47,008::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-181::DEBUG::2014-10-23 17:15:47,009::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-181::DEBUG::2014-10-23 17:15:47,009::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-181::DEBUG::2014-10-23 17:15:47,009::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-181::DEBUG::2014-10-23 17:15:47,009::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-181::WARNING::2014-10-23 17:15:47,009::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/00000002-0002-0002-0002-000000000067 already exists >Thread-181::DEBUG::2014-10-23 17:15:47,009::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:47,009::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-181::DEBUG::2014-10-23 17:15:47,113::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-181::DEBUG::2014-10-23 17:15:47,113::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-181::DEBUG::2014-10-23 17:15:47,113::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-181::WARNING::2014-10-23 17:15:47,114::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-181::DEBUG::2014-10-23 17:15:47,114::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-181::WARNING::2014-10-23 17:15:47,114::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-181::INFO::2014-10-23 17:15:47,114::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-181::DEBUG::2014-10-23 17:15:47,114::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-181::INFO::2014-10-23 17:15:47,114::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-181::DEBUG::2014-10-23 17:15:47,114::sp::1131::Storage.StoragePool::(_linkStorageDomain) link already present skipping creation for /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-181::DEBUG::2014-10-23 17:15:47,114::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-181::DEBUG::2014-10-23 17:15:47,115::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-181::INFO::2014-10-23 17:15:47,115::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-181::DEBUG::2014-10-23 17:15:47,115::task::1191::Storage.TaskManager.Task::(prepare) Task=`87875458-65ff-4166-9f79-b6328fb16b53`::finished: True >Thread-181::DEBUG::2014-10-23 17:15:47,116::task::595::Storage.TaskManager.Task::(_updateState) Task=`87875458-65ff-4166-9f79-b6328fb16b53`::moving from state preparing -> state finished >Thread-181::DEBUG::2014-10-23 17:15:47,116::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-181::DEBUG::2014-10-23 17:15:47,116::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-181::DEBUG::2014-10-23 17:15:47,116::task::993::Storage.TaskManager.Task::(_decref) Task=`87875458-65ff-4166-9f79-b6328fb16b53`::ref 0 aborting False >Thread-181::DEBUG::2014-10-23 17:15:47,116::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-181::DEBUG::2014-10-23 17:15:47,116::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:47,189::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:47,189::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-182::DEBUG::2014-10-23 17:15:47,190::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:47,219::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:47,219::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-183::DEBUG::2014-10-23 17:15:47,219::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnectStorageServer' in bridge with {u'connectionParams': [], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-183::DEBUG::2014-10-23 17:15:47,220::task::595::Storage.TaskManager.Task::(_updateState) Task=`14ca9558-9726-4884-b860-166986860f3f`::moving from state init -> state preparing >Thread-183::INFO::2014-10-23 17:15:47,220::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[], options=None) >Thread-183::DEBUG::2014-10-23 17:15:47,221::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-183::DEBUG::2014-10-23 17:15:47,221::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-183::DEBUG::2014-10-23 17:15:47,221::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-183::DEBUG::2014-10-23 17:15:47,221::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-183::DEBUG::2014-10-23 17:15:47,221::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-183::DEBUG::2014-10-23 17:15:47,221::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-183::DEBUG::2014-10-23 17:15:47,234::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-183::DEBUG::2014-10-23 17:15:47,234::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-183::DEBUG::2014-10-23 17:15:47,298::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-183::DEBUG::2014-10-23 17:15:47,299::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-183::INFO::2014-10-23 17:15:47,300::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': []} >Thread-183::DEBUG::2014-10-23 17:15:47,300::task::1191::Storage.TaskManager.Task::(prepare) Task=`14ca9558-9726-4884-b860-166986860f3f`::finished: {'statuslist': []} >Thread-183::DEBUG::2014-10-23 17:15:47,300::task::595::Storage.TaskManager.Task::(_updateState) Task=`14ca9558-9726-4884-b860-166986860f3f`::moving from state preparing -> state finished >Thread-183::DEBUG::2014-10-23 17:15:47,300::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-183::DEBUG::2014-10-23 17:15:47,300::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-183::DEBUG::2014-10-23 17:15:47,300::task::993::Storage.TaskManager.Task::(_decref) Task=`14ca9558-9726-4884-b860-166986860f3f`::ref 0 aborting False >Thread-183::DEBUG::2014-10-23 17:15:47,300::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnectStorageServer' in bridge with [] >Thread-183::DEBUG::2014-10-23 17:15:47,300::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:15:47,785::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:47,875::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:47,881::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:47,881::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:47,891::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.00436499 s, 938 kB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:47,893::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-72::DEBUG::2014-10-23 17:15:47,894::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:47,925::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0202341 s, 50.6 MB/s\n'; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:47,988::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-72::DEBUG::2014-10-23 17:15:47,988::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-114::DEBUG::2014-10-23 17:15:47,990::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:47,990::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:47,991::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:47,991::__init__::232::IOProcess::(_processLogs) (29) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:47,991::__init__::232::IOProcess::(_processLogs) (29) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:47,991::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:47,991::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:47,992::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:47,992::__init__::232::IOProcess::(_processLogs) (30) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:47,992::__init__::232::IOProcess::(_processLogs) (30) Queuing response >Dummy-120::DEBUG::2014-10-23 17:15:49,934::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:49,975::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.012495 s, 82.0 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:50,209::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:50,210::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-184::DEBUG::2014-10-23 17:15:50,211::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:51,863::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:51,864::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-185::DEBUG::2014-10-23 17:15:51,864::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-185::DEBUG::2014-10-23 17:15:51,865::task::595::Storage.TaskManager.Task::(_updateState) Task=`534dd97b-e30e-4223-80e9-14d02066df41`::moving from state init -> state preparing >Thread-185::INFO::2014-10-23 17:15:51,865::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-185::INFO::2014-10-23 17:15:51,877::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-185::DEBUG::2014-10-23 17:15:51,877::task::1191::Storage.TaskManager.Task::(prepare) Task=`534dd97b-e30e-4223-80e9-14d02066df41`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-185::DEBUG::2014-10-23 17:15:51,877::task::595::Storage.TaskManager.Task::(_updateState) Task=`534dd97b-e30e-4223-80e9-14d02066df41`::moving from state preparing -> state finished >Thread-185::DEBUG::2014-10-23 17:15:51,877::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-185::DEBUG::2014-10-23 17:15:51,877::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-185::DEBUG::2014-10-23 17:15:51,877::task::993::Storage.TaskManager.Task::(_decref) Task=`534dd97b-e30e-4223-80e9-14d02066df41`::ref 0 aborting False >Thread-185::DEBUG::2014-10-23 17:15:51,877::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-185::DEBUG::2014-10-23 17:15:51,878::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:51,885::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:51,886::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-186::DEBUG::2014-10-23 17:15:51,886::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-186::DEBUG::2014-10-23 17:15:51,886::task::595::Storage.TaskManager.Task::(_updateState) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::moving from state init -> state preparing >Thread-186::INFO::2014-10-23 17:15:51,886::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-186::DEBUG::2014-10-23 17:15:51,887::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`d80ae35e-1bba-410d-843c-c135f409572c`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-186::DEBUG::2014-10-23 17:15:51,887::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-186::DEBUG::2014-10-23 17:15:51,887::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-186::DEBUG::2014-10-23 17:15:51,887::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`d80ae35e-1bba-410d-843c-c135f409572c`::Granted request >Thread-186::DEBUG::2014-10-23 17:15:51,887::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-186::DEBUG::2014-10-23 17:15:51,887::task::993::Storage.TaskManager.Task::(_decref) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::ref 1 aborting False >Thread-186::INFO::2014-10-23 17:15:51,898::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-186::DEBUG::2014-10-23 17:15:51,898::task::1191::Storage.TaskManager.Task::(prepare) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-186::DEBUG::2014-10-23 17:15:51,898::task::595::Storage.TaskManager.Task::(_updateState) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::moving from state preparing -> state finished >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-186::DEBUG::2014-10-23 17:15:51,899::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-186::DEBUG::2014-10-23 17:15:51,899::task::993::Storage.TaskManager.Task::(_decref) Task=`f325b0e0-35bf-4e99-962a-29b93e13e49b`::ref 0 aborting False >Thread-186::DEBUG::2014-10-23 17:15:51,899::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-186::DEBUG::2014-10-23 17:15:51,900::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:51,984::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:52,024::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.011951 s, 85.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:53,234::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:53,235::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-187::DEBUG::2014-10-23 17:15:53,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`d16d47bf-9c30-48ad-8a2f-cf3af8b62b53`::moving from state init -> state preparing >Thread-187::INFO::2014-10-23 17:15:53,236::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-187::INFO::2014-10-23 17:15:53,236::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00436499', 'lastCheck': '5.2', 'valid': True}} >Thread-187::DEBUG::2014-10-23 17:15:53,236::task::1191::Storage.TaskManager.Task::(prepare) Task=`d16d47bf-9c30-48ad-8a2f-cf3af8b62b53`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.00436499', 'lastCheck': '5.2', 'valid': True}} >Thread-187::DEBUG::2014-10-23 17:15:53,236::task::595::Storage.TaskManager.Task::(_updateState) Task=`d16d47bf-9c30-48ad-8a2f-cf3af8b62b53`::moving from state preparing -> state finished >Thread-187::DEBUG::2014-10-23 17:15:53,236::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-187::DEBUG::2014-10-23 17:15:53,237::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-187::DEBUG::2014-10-23 17:15:53,237::task::993::Storage.TaskManager.Task::(_decref) Task=`d16d47bf-9c30-48ad-8a2f-cf3af8b62b53`::ref 0 aborting False >Thread-187::DEBUG::2014-10-23 17:15:53,238::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:53,311::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:53,312::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-188::DEBUG::2014-10-23 17:15:53,312::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:15:54,033::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:54,075::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0133088 s, 76.9 MB/s\n'; <rc> = 0 >Dummy-120::DEBUG::2014-10-23 17:15:56,084::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:56,126::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0124592 s, 82.2 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:56,518::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:56,519::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-189::DEBUG::2014-10-23 17:15:56,521::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:15:57,993::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-72::DEBUG::2014-10-23 17:15:57,998::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000541736 s, 7.6 MB/s\n'; <rc> = 0 >Thread-114::DEBUG::2014-10-23 17:15:57,999::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:58,000::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:58,000::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) (31) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) (31) Queuing response >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:15:58,001::__init__::232::IOProcess::(_processLogs) (32) Got request for method 'access' >Thread-114::DEBUG::2014-10-23 17:15:58,002::__init__::232::IOProcess::(_processLogs) (32) Queuing response >Dummy-120::DEBUG::2014-10-23 17:15:58,135::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:15:58,176::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0130385 s, 78.5 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:15:59,532::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:15:59,533::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-190::DEBUG::2014-10-23 17:15:59,534::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:16:00,185::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:16:00,234::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0174161 s, 58.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:01,933::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:01,934::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-191::DEBUG::2014-10-23 17:16:01,934::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-191::DEBUG::2014-10-23 17:16:01,935::task::595::Storage.TaskManager.Task::(_updateState) Task=`9653ef15-7064-458d-833b-98b76a394ecb`::moving from state init -> state preparing >Thread-191::INFO::2014-10-23 17:16:01,935::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-191::INFO::2014-10-23 17:16:01,947::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-191::DEBUG::2014-10-23 17:16:01,947::task::1191::Storage.TaskManager.Task::(prepare) Task=`9653ef15-7064-458d-833b-98b76a394ecb`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L}} >Thread-191::DEBUG::2014-10-23 17:16:01,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`9653ef15-7064-458d-833b-98b76a394ecb`::moving from state preparing -> state finished >Thread-191::DEBUG::2014-10-23 17:16:01,947::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-191::DEBUG::2014-10-23 17:16:01,947::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-191::DEBUG::2014-10-23 17:16:01,947::task::993::Storage.TaskManager.Task::(_decref) Task=`9653ef15-7064-458d-833b-98b76a394ecb`::ref 0 aborting False >Thread-191::DEBUG::2014-10-23 17:16:01,947::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 2L} >Thread-191::DEBUG::2014-10-23 17:16:01,947::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:01,957::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:01,958::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-192::DEBUG::2014-10-23 17:16:01,958::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getInfo' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-192::DEBUG::2014-10-23 17:16:01,959::task::595::Storage.TaskManager.Task::(_updateState) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::moving from state init -> state preparing >Thread-192::INFO::2014-10-23 17:16:01,959::logUtils::44::dispatcher::(wrapper) Run and protect: getStoragePoolInfo(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-192::DEBUG::2014-10-23 17:16:01,960::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a1dc93d7-485c-42d9-b0fd-24e05edd0ec3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '2616' at 'getStoragePoolInfo' >Thread-192::DEBUG::2014-10-23 17:16:01,960::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'shared' >Thread-192::DEBUG::2014-10-23 17:16:01,960::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'shared' (1 active user) >Thread-192::DEBUG::2014-10-23 17:16:01,960::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`a1dc93d7-485c-42d9-b0fd-24e05edd0ec3`::Granted request >Thread-192::DEBUG::2014-10-23 17:16:01,960::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (shared) >Thread-192::DEBUG::2014-10-23 17:16:01,960::task::993::Storage.TaskManager.Task::(_decref) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::ref 1 aborting False >Thread-192::INFO::2014-10-23 17:16:01,972::logUtils::47::dispatcher::(wrapper) Run and protect: getStoragePoolInfo, Return response: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-192::DEBUG::2014-10-23 17:16:01,972::task::1191::Storage.TaskManager.Task::(prepare) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::finished: {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-192::DEBUG::2014-10-23 17:16:01,972::task::595::Storage.TaskManager.Task::(_updateState) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::moving from state preparing -> state finished >Thread-192::DEBUG::2014-10-23 17:16:01,972::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-192::DEBUG::2014-10-23 17:16:01,973::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-192::DEBUG::2014-10-23 17:16:01,973::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-192::DEBUG::2014-10-23 17:16:01,973::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-192::DEBUG::2014-10-23 17:16:01,973::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-192::DEBUG::2014-10-23 17:16:01,973::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-192::DEBUG::2014-10-23 17:16:01,973::task::993::Storage.TaskManager.Task::(_decref) Task=`de2a8a0e-df90-4f1d-8e42-964f6df6f859`::ref 0 aborting False >Thread-192::DEBUG::2014-10-23 17:16:01,973::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getInfo' in bridge with {'info': {'name': 'No Description', 'isoprefix': '', 'pool_status': 'connected', 'lver': 2L, 'domains': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'master_uuid': '5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'version': '3', 'spm_id': 1, 'type': 'ISCSI', 'master_ver': 1}, 'dominfo': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'status': u'Active', 'diskfree': '86704652288', 'isoprefix': '', 'alerts': [], 'disktotal': '90865401856', 'version': 3}}} >Thread-192::DEBUG::2014-10-23 17:16:01,974::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:16:02,252::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:16:02,293::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0120709 s, 84.8 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:02,547::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:02,548::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-193::DEBUG::2014-10-23 17:16:02,549::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-120::DEBUG::2014-10-23 17:16:04,302::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >Dummy-120::DEBUG::2014-10-23 17:16:04,350::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.019795 s, 51.7 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:05,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:05,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-194::DEBUG::2014-10-23 17:16:05,573::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:05,946::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:05,946::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-195::DEBUG::2014-10-23 17:16:05,946::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StorageDomain.deactivate' in bridge with {u'masterVersion': 1, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'00000000-0000-0000-0000-000000000000', u'storagedomainID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f'} >Thread-195::DEBUG::2014-10-23 17:16:05,947::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::moving from state init -> state preparing >Thread-195::INFO::2014-10-23 17:16:05,948::logUtils::44::dispatcher::(wrapper) Run and protect: deactivateStorageDomain(sdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', spUUID=u'00000002-0002-0002-0002-000000000067', msdUUID=u'00000000-0000-0000-0000-000000000000', masterVersion=1, options=None) >Thread-195::DEBUG::2014-10-23 17:16:05,948::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`89ca42ab-01f7-4bc4-a6b0-9e78cacdf782`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1227' at 'deactivateStorageDomain' >Thread-195::DEBUG::2014-10-23 17:16:05,948::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-195::DEBUG::2014-10-23 17:16:05,948::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-195::DEBUG::2014-10-23 17:16:05,948::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`89ca42ab-01f7-4bc4-a6b0-9e78cacdf782`::Granted request >Thread-195::DEBUG::2014-10-23 17:16:05,948::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-195::DEBUG::2014-10-23 17:16:05,949::task::993::Storage.TaskManager.Task::(_decref) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::ref 1 aborting False >Thread-195::DEBUG::2014-10-23 17:16:05,949::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`1b6f7c20-9780-483c-8d44-5c9b55d12de6`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1228' at 'deactivateStorageDomain' >Thread-195::DEBUG::2014-10-23 17:16:05,949::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >Thread-195::DEBUG::2014-10-23 17:16:05,949::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >Thread-195::DEBUG::2014-10-23 17:16:05,949::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`1b6f7c20-9780-483c-8d44-5c9b55d12de6`::Granted request >Thread-195::DEBUG::2014-10-23 17:16:05,949::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::_resourcesAcquired: Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (exclusive) >Thread-195::DEBUG::2014-10-23 17:16:05,949::task::993::Storage.TaskManager.Task::(_decref) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::ref 1 aborting False >Thread-195::INFO::2014-10-23 17:16:05,949::sp::1070::Storage.StoragePool::(deactivateSD) sdUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f spUUID=00000002-0002-0002-0002-000000000067 newMsdUUID=00000000-0000-0000-0000-000000000000 >Thread-195::INFO::2014-10-23 17:16:05,950::sp::1094::Storage.StoragePool::(deactivateSD) Silently ignoring the deactivation request for the master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-195::INFO::2014-10-23 17:16:05,950::logUtils::47::dispatcher::(wrapper) Run and protect: deactivateStorageDomain, Return response: None >Thread-195::DEBUG::2014-10-23 17:16:05,950::task::1191::Storage.TaskManager.Task::(prepare) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::finished: None >Thread-195::DEBUG::2014-10-23 17:16:05,950::task::595::Storage.TaskManager.Task::(_updateState) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::moving from state preparing -> state finished >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': < ResourceRef 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', isValid: 'True' obj: 'None'>, u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >Thread-195::DEBUG::2014-10-23 17:16:05,950::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >Thread-195::DEBUG::2014-10-23 17:16:05,951::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-195::DEBUG::2014-10-23 17:16:05,951::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-195::DEBUG::2014-10-23 17:16:05,951::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-195::DEBUG::2014-10-23 17:16:05,951::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-195::DEBUG::2014-10-23 17:16:05,951::task::993::Storage.TaskManager.Task::(_decref) Task=`4f730be1-a8cc-4e6a-aa0e-d367dd04abbc`::ref 0 aborting False >Thread-195::DEBUG::2014-10-23 17:16:05,951::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StorageDomain.deactivate' in bridge with True >Thread-195::DEBUG::2014-10-23 17:16:05,951::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:05,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:05,963::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-196::DEBUG::2014-10-23 17:16:05,964::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Host.getAllTasksStatuses' in bridge with {} >Thread-196::DEBUG::2014-10-23 17:16:05,964::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccde01cc-815f-423c-9b66-ace653b83fd5`::moving from state init -> state preparing >Thread-196::INFO::2014-10-23 17:16:05,964::logUtils::44::dispatcher::(wrapper) Run and protect: getAllTasksStatuses(spUUID=None, options=None) >Thread-196::DEBUG::2014-10-23 17:16:05,965::taskManager::112::Storage.TaskManager::(getAllTasksStatuses) Entry. >Thread-196::DEBUG::2014-10-23 17:16:05,965::taskManager::123::Storage.TaskManager::(getAllTasksStatuses) Return: {} >Thread-196::INFO::2014-10-23 17:16:05,965::logUtils::47::dispatcher::(wrapper) Run and protect: getAllTasksStatuses, Return response: {'allTasksStatus': {}} >Thread-196::DEBUG::2014-10-23 17:16:05,965::task::1191::Storage.TaskManager.Task::(prepare) Task=`ccde01cc-815f-423c-9b66-ace653b83fd5`::finished: {'allTasksStatus': {}} >Thread-196::DEBUG::2014-10-23 17:16:05,965::task::595::Storage.TaskManager.Task::(_updateState) Task=`ccde01cc-815f-423c-9b66-ace653b83fd5`::moving from state preparing -> state finished >Thread-196::DEBUG::2014-10-23 17:16:05,965::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-196::DEBUG::2014-10-23 17:16:05,965::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-196::DEBUG::2014-10-23 17:16:05,965::task::993::Storage.TaskManager.Task::(_decref) Task=`ccde01cc-815f-423c-9b66-ace653b83fd5`::ref 0 aborting False >Thread-196::DEBUG::2014-10-23 17:16:05,965::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Host.getAllTasksStatuses' in bridge with {} >Thread-196::DEBUG::2014-10-23 17:16:05,965::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:05,974::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:05,974::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-197::DEBUG::2014-10-23 17:16:05,974::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.spmStop' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-197::DEBUG::2014-10-23 17:16:05,975::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::moving from state init -> state preparing >Thread-197::INFO::2014-10-23 17:16:05,975::logUtils::44::dispatcher::(wrapper) Run and protect: spmStop(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-197::DEBUG::2014-10-23 17:16:05,975::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`100d946b-d524-44f8-9751-ec28d91719be`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '596' at 'spmStop' >Thread-197::DEBUG::2014-10-23 17:16:05,976::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-197::DEBUG::2014-10-23 17:16:05,976::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-197::DEBUG::2014-10-23 17:16:05,976::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`100d946b-d524-44f8-9751-ec28d91719be`::Granted request >Thread-197::DEBUG::2014-10-23 17:16:05,976::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-197::DEBUG::2014-10-23 17:16:05,976::task::993::Storage.TaskManager.Task::(_decref) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::ref 1 aborting False >Thread-197::DEBUG::2014-10-23 17:16:05,976::sp::337::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-197::DEBUG::2014-10-23 17:16:05,976::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`ad61a556-77ac-4e9d-8e5d-7a12ea204b63`::Request was made in '/usr/share/vdsm/storage/sp.py' line '339' at '_shutDownUpgrade' >Thread-197::DEBUG::2014-10-23 17:16:05,976::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`ad61a556-77ac-4e9d-8e5d-7a12ea204b63`::Granted request >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' (0 active users) >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-197::DEBUG::2014-10-23 17:16:05,977::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067', Clearing records. >Thread-197::DEBUG::2014-10-23 17:16:05,978::sp::375::Storage.StoragePool::(cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master >Thread-197::DEBUG::2014-10-23 17:16:05,997::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-197::INFO::2014-10-23 17:16:06,021::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-197::DEBUG::2014-10-23 17:16:06,052::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully released >Thread-197::INFO::2014-10-23 17:16:06,052::logUtils::47::dispatcher::(wrapper) Run and protect: spmStop, Return response: None >Thread-197::DEBUG::2014-10-23 17:16:06,052::task::1191::Storage.TaskManager.Task::(prepare) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::finished: None >Thread-197::DEBUG::2014-10-23 17:16:06,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::moving from state preparing -> state finished >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-197::DEBUG::2014-10-23 17:16:06,053::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-197::DEBUG::2014-10-23 17:16:06,053::task::993::Storage.TaskManager.Task::(_decref) Task=`a8f125a4-8dd3-4d24-b58e-e2203d96d67e`::ref 0 aborting False >Thread-197::DEBUG::2014-10-23 17:16:06,054::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.spmStop' in bridge with True >Thread-197::DEBUG::2014-10-23 17:16:06,054::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:06,072::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:06,073::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-198::DEBUG::2014-10-23 17:16:06,073::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnect' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'hostID': 1} >Thread-198::DEBUG::2014-10-23 17:16:06,074::task::595::Storage.TaskManager.Task::(_updateState) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::moving from state init -> state preparing >Thread-198::INFO::2014-10-23 17:16:06,075::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, remove=False, options=None) >Thread-198::DEBUG::2014-10-23 17:16:06,075::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`4b66a275-b1b3-46fa-8840-bdabe113a859`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1124' at 'disconnectStoragePool' >Thread-198::DEBUG::2014-10-23 17:16:06,075::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-198::DEBUG::2014-10-23 17:16:06,075::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-198::DEBUG::2014-10-23 17:16:06,075::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`4b66a275-b1b3-46fa-8840-bdabe113a859`::Granted request >Thread-198::DEBUG::2014-10-23 17:16:06,076::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-198::DEBUG::2014-10-23 17:16:06,076::task::993::Storage.TaskManager.Task::(_decref) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::ref 1 aborting False >Thread-198::DEBUG::2014-10-23 17:16:06,076::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`8f5757de-6f52-4a1b-9ef8-c9c06fc19387`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1132' at '_disconnectPool' >Thread-198::DEBUG::2014-10-23 17:16:06,076::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-198::DEBUG::2014-10-23 17:16:06,076::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-198::DEBUG::2014-10-23 17:16:06,076::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`8f5757de-6f52-4a1b-9ef8-c9c06fc19387`::Granted request >Thread-198::INFO::2014-10-23 17:16:06,077::sp::657::Storage.StoragePool::(disconnect) Disconnect from the storage pool 00000002-0002-0002-0002-000000000067 >Thread-198::DEBUG::2014-10-23 17:16:07,046::fileUtils::124::Storage.fileUtils::(cleanupdir) Removing directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-198::INFO::2014-10-23 17:16:07,046::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-72::DEBUG::2014-10-23 17:16:07,046::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-72::INFO::2014-10-23 17:16:07,046::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >Dummy-120::INFO::2014-10-23 17:16:07,690::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-80::INFO::2014-10-23 17:16:08,076::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-80::INFO::2014-10-23 17:16:08,077::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-80::DEBUG::2014-10-23 17:16:08,077::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-80::DEBUG::2014-10-23 17:16:08,081::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) FAILED: <err> = "/bin/dd: opening `/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox': No such file or directory\n"; <rc> = 1 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:08,591::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:08,592::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-199::DEBUG::2014-10-23 17:16:08,593::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02ac633-0ef6-4dbe-80d2-7b742240aa89`::moving from state init -> state preparing >Thread-199::INFO::2014-10-23 17:16:08,594::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-199::INFO::2014-10-23 17:16:08,594::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000541736', 'lastCheck': '10.6', 'valid': True}} >Thread-199::DEBUG::2014-10-23 17:16:08,594::task::1191::Storage.TaskManager.Task::(prepare) Task=`d02ac633-0ef6-4dbe-80d2-7b742240aa89`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': True, 'delay': '0.000541736', 'lastCheck': '10.6', 'valid': True}} >Thread-199::DEBUG::2014-10-23 17:16:08,594::task::595::Storage.TaskManager.Task::(_updateState) Task=`d02ac633-0ef6-4dbe-80d2-7b742240aa89`::moving from state preparing -> state finished >Thread-199::DEBUG::2014-10-23 17:16:08,594::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-199::DEBUG::2014-10-23 17:16:08,594::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-199::DEBUG::2014-10-23 17:16:08,594::task::993::Storage.TaskManager.Task::(_decref) Task=`d02ac633-0ef6-4dbe-80d2-7b742240aa89`::ref 0 aborting False >Thread-199::DEBUG::2014-10-23 17:16:08,596::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:08,638::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:08,639::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-200::DEBUG::2014-10-23 17:16:08,639::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-72::DEBUG::2014-10-23 17:16:09,051::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f released successfully (id: 1) >Thread-198::DEBUG::2014-10-23 17:16:09,051::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-198::INFO::2014-10-23 17:16:09,052::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: True >Thread-198::DEBUG::2014-10-23 17:16:09,052::task::1191::Storage.TaskManager.Task::(prepare) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::finished: True >Thread-198::DEBUG::2014-10-23 17:16:09,052::task::595::Storage.TaskManager.Task::(_updateState) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::moving from state preparing -> state finished >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-198::DEBUG::2014-10-23 17:16:09,052::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-198::DEBUG::2014-10-23 17:16:09,053::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-198::DEBUG::2014-10-23 17:16:09,053::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-198::DEBUG::2014-10-23 17:16:09,053::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-198::DEBUG::2014-10-23 17:16:09,053::task::993::Storage.TaskManager.Task::(_decref) Task=`6315dd88-0faf-487f-bb1f-18f110155383`::ref 0 aborting False >Thread-198::DEBUG::2014-10-23 17:16:09,053::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnect' in bridge with True >Thread-198::DEBUG::2014-10-23 17:16:09,053::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:09,060::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:09,061::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-201::DEBUG::2014-10-23 17:16:09,061::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnect' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'hostID': 1} >Thread-201::DEBUG::2014-10-23 17:16:09,061::task::595::Storage.TaskManager.Task::(_updateState) Task=`e07d6778-16d4-4dbd-bb55-e0963a957bf2`::moving from state init -> state preparing >Thread-201::INFO::2014-10-23 17:16:09,062::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, remove=False, options=None) >Thread-201::WARNING::2014-10-23 17:16:09,062::hsm::1119::Storage.HSM::(disconnectStoragePool) disconnect sp: 00000002-0002-0002-0002-000000000067 failed. Known pools {} >Thread-201::INFO::2014-10-23 17:16:09,062::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: None >Thread-201::DEBUG::2014-10-23 17:16:09,062::task::1191::Storage.TaskManager.Task::(prepare) Task=`e07d6778-16d4-4dbd-bb55-e0963a957bf2`::finished: None >Thread-201::DEBUG::2014-10-23 17:16:09,062::task::595::Storage.TaskManager.Task::(_updateState) Task=`e07d6778-16d4-4dbd-bb55-e0963a957bf2`::moving from state preparing -> state finished >Thread-201::DEBUG::2014-10-23 17:16:09,062::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-201::DEBUG::2014-10-23 17:16:09,062::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-201::DEBUG::2014-10-23 17:16:09,062::task::993::Storage.TaskManager.Task::(_decref) Task=`e07d6778-16d4-4dbd-bb55-e0963a957bf2`::ref 0 aborting False >Thread-201::DEBUG::2014-10-23 17:16:09,062::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnect' in bridge with True >Thread-201::DEBUG::2014-10-23 17:16:09,062::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:09,105::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:09,106::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-202::DEBUG::2014-10-23 17:16:09,106::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-202::DEBUG::2014-10-23 17:16:09,107::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0cd3cad-4615-494d-9224-13e9ea08ebf1`::moving from state init -> state preparing >Thread-202::INFO::2014-10-23 17:16:09,107::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-202::DEBUG::2014-10-23 17:16:09,119::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -r 4 -u (cwd None) >Thread-202::DEBUG::2014-10-23 17:16:09,719::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-202::DEBUG::2014-10-23 17:16:09,719::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-202::DEBUG::2014-10-23 17:16:09,719::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-202::DEBUG::2014-10-23 17:16:09,719::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-202::DEBUG::2014-10-23 17:16:09,719::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-202::DEBUG::2014-10-23 17:16:09,719::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-202::DEBUG::2014-10-23 17:16:09,720::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-202::DEBUG::2014-10-23 17:16:09,728::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-202::DEBUG::2014-10-23 17:16:09,729::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-202::DEBUG::2014-10-23 17:16:09,841::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-202::DEBUG::2014-10-23 17:16:09,841::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,841::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,842::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,842::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,842::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,842::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-202::DEBUG::2014-10-23 17:16:09,842::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-202::INFO::2014-10-23 17:16:09,842::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-202::DEBUG::2014-10-23 17:16:09,843::task::1191::Storage.TaskManager.Task::(prepare) Task=`c0cd3cad-4615-494d-9224-13e9ea08ebf1`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-202::DEBUG::2014-10-23 17:16:09,843::task::595::Storage.TaskManager.Task::(_updateState) Task=`c0cd3cad-4615-494d-9224-13e9ea08ebf1`::moving from state preparing -> state finished >Thread-202::DEBUG::2014-10-23 17:16:09,843::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-202::DEBUG::2014-10-23 17:16:09,843::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-202::DEBUG::2014-10-23 17:16:09,843::task::993::Storage.TaskManager.Task::(_decref) Task=`c0cd3cad-4615-494d-9224-13e9ea08ebf1`::ref 0 aborting False >Thread-202::DEBUG::2014-10-23 17:16:09,843::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-202::DEBUG::2014-10-23 17:16:09,843::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:09,866::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:09,867::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-203::DEBUG::2014-10-23 17:16:09,867::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-203::DEBUG::2014-10-23 17:16:09,868::task::595::Storage.TaskManager.Task::(_updateState) Task=`b33aeedc-76dc-4a62-8f17-803a6fbe9930`::moving from state init -> state preparing >Thread-203::INFO::2014-10-23 17:16:09,868::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-203::DEBUG::2014-10-23 17:16:09,869::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-203::DEBUG::2014-10-23 17:16:09,869::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-203::DEBUG::2014-10-23 17:16:09,869::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-203::DEBUG::2014-10-23 17:16:09,869::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-203::DEBUG::2014-10-23 17:16:09,869::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-203::DEBUG::2014-10-23 17:16:09,870::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-203::DEBUG::2014-10-23 17:16:09,878::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-203::DEBUG::2014-10-23 17:16:09,878::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-203::DEBUG::2014-10-23 17:16:09,932::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-203::DEBUG::2014-10-23 17:16:09,932::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-203::DEBUG::2014-10-23 17:16:09,933::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-203::INFO::2014-10-23 17:16:09,933::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-203::DEBUG::2014-10-23 17:16:09,933::task::1191::Storage.TaskManager.Task::(prepare) Task=`b33aeedc-76dc-4a62-8f17-803a6fbe9930`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-203::DEBUG::2014-10-23 17:16:09,934::task::595::Storage.TaskManager.Task::(_updateState) Task=`b33aeedc-76dc-4a62-8f17-803a6fbe9930`::moving from state preparing -> state finished >Thread-203::DEBUG::2014-10-23 17:16:09,934::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-203::DEBUG::2014-10-23 17:16:09,934::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-203::DEBUG::2014-10-23 17:16:09,934::task::993::Storage.TaskManager.Task::(_decref) Task=`b33aeedc-76dc-4a62-8f17-803a6fbe9930`::ref 0 aborting False >Thread-203::DEBUG::2014-10-23 17:16:09,934::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-203::DEBUG::2014-10-23 17:16:09,934::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:11,814::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:11,816::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-204::DEBUG::2014-10-23 17:16:11,819::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:14,836::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:14,836::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-205::DEBUG::2014-10-23 17:16:14,838::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:17,868::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:17,869::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-206::DEBUG::2014-10-23 17:16:17,870::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:18,317::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:18,319::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-207::DEBUG::2014-10-23 17:16:18,319::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-207::DEBUG::2014-10-23 17:16:18,320::task::595::Storage.TaskManager.Task::(_updateState) Task=`78d42db7-9f33-4463-9715-328b54a7b93b`::moving from state init -> state preparing >Thread-207::INFO::2014-10-23 17:16:18,320::logUtils::44::dispatcher::(wrapper) Run and protect: connectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-207::DEBUG::2014-10-23 17:16:18,320::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 --op=new (cwd None) >Thread-207::DEBUG::2014-10-23 17:16:18,331::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-207::DEBUG::2014-10-23 17:16:18,331::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -l (cwd None) >Thread-207::DEBUG::2014-10-23 17:16:18,861::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-207::DEBUG::2014-10-23 17:16:18,862::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m node -T iqn.2012-10.brq.str-01:cmestreg -I default -p 10.34.63.202:3260,1 -n node.startup -v manual --op=update (cwd None) >Thread-207::DEBUG::2014-10-23 17:16:18,879::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-207::DEBUG::2014-10-23 17:16:18,879::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-207::DEBUG::2014-10-23 17:16:18,881::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-207::DEBUG::2014-10-23 17:16:19,025::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-207::DEBUG::2014-10-23 17:16:19,025::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-207::DEBUG::2014-10-23 17:16:19,026::hsm::2396::Storage.HSM::(__prefetchDomains) Found SD uuids: ('5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'dae2b061-ea53-4ca2-8b91-f24f0641dc3f') >Thread-207::DEBUG::2014-10-23 17:16:19,026::hsm::2452::Storage.HSM::(connectStorageServer) knownSDs: {5c8c20ff-41e1-4a43-91a8-fbffa2f8334f: storage.blockSD.findDomain, dae2b061-ea53-4ca2-8b91-f24f0641dc3f: storage.blockSD.findDomain} >Thread-207::INFO::2014-10-23 17:16:19,026::logUtils::47::dispatcher::(wrapper) Run and protect: connectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-207::DEBUG::2014-10-23 17:16:19,028::task::1191::Storage.TaskManager.Task::(prepare) Task=`78d42db7-9f33-4463-9715-328b54a7b93b`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-207::DEBUG::2014-10-23 17:16:19,028::task::595::Storage.TaskManager.Task::(_updateState) Task=`78d42db7-9f33-4463-9715-328b54a7b93b`::moving from state preparing -> state finished >Thread-207::DEBUG::2014-10-23 17:16:19,029::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-207::DEBUG::2014-10-23 17:16:19,029::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-207::DEBUG::2014-10-23 17:16:19,029::task::993::Storage.TaskManager.Task::(_decref) Task=`78d42db7-9f33-4463-9715-328b54a7b93b`::ref 0 aborting False >Thread-207::DEBUG::2014-10-23 17:16:19,029::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-207::DEBUG::2014-10-23 17:16:19,029::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:19,046::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:19,047::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-208::DEBUG::2014-10-23 17:16:19,047::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.connect' in bridge with {u'masterVersion': 1, u'domainDict': {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'masterSdUUID': u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', u'hostID': 1} >Thread-208::DEBUG::2014-10-23 17:16:19,048::task::595::Storage.TaskManager.Task::(_updateState) Task=`3979b6ec-8288-49f1-b7b5-d65861623372`::moving from state init -> state preparing >Thread-208::INFO::2014-10-23 17:16:19,048::logUtils::44::dispatcher::(wrapper) Run and protect: connectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, msdUUID=u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', masterVersion=1, domainsMap={u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'}, options=None) >Thread-208::DEBUG::2014-10-23 17:16:19,048::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`dbc9fabd-ecd4-4bab-9c28-ac2a53d71690`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1011' at 'connectStoragePool' >Thread-208::DEBUG::2014-10-23 17:16:19,048::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-208::DEBUG::2014-10-23 17:16:19,048::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-208::DEBUG::2014-10-23 17:16:19,048::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`dbc9fabd-ecd4-4bab-9c28-ac2a53d71690`::Granted request >Thread-208::DEBUG::2014-10-23 17:16:19,049::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`1b044c2a-8abb-40f3-909a-0021284b2533`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1059' at '_connectStoragePool' >Thread-208::DEBUG::2014-10-23 17:16:19,049::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-208::DEBUG::2014-10-23 17:16:19,049::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-208::DEBUG::2014-10-23 17:16:19,049::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`1b044c2a-8abb-40f3-909a-0021284b2533`::Granted request >Thread-208::DEBUG::2014-10-23 17:16:19,049::spbackends::485::Storage.StoragePoolMemoryBackend::(updateVersionAndDomains) updating domain version to 1 and domains map to {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'active'} >Thread-208::INFO::2014-10-23 17:16:19,049::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': u'Active'} >Thread-208::INFO::2014-10-23 17:16:19,049::sp::133::Storage.StoragePool::(setBackend) updating pool 00000002-0002-0002-0002-000000000067 backend from type NoneType instance 0x3533572210 to type StoragePoolMemoryBackend instance 0x7fa05c462f70 >Thread-208::INFO::2014-10-23 17:16:19,050::sp::631::Storage.StoragePool::(connect) Connect host #1 to the storage pool 00000002-0002-0002-0002-000000000067 with master domain: 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (ver = 1) >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,050::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-208::DEBUG::2014-10-23 17:16:19,050::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-208::DEBUG::2014-10-23 17:16:19,050::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-208::DEBUG::2014-10-23 17:16:19,051::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-208::DEBUG::2014-10-23 17:16:19,051::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-208::DEBUG::2014-10-23 17:16:19,051::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,066::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-208::DEBUG::2014-10-23 17:16:19,066::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,154::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,154::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,155::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-208::DEBUG::2014-10-23 17:16:19,155::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,156::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,253::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,253::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,254::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >Thread-208::DEBUG::2014-10-23 17:16:19,254::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,259::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000557027 s, 3.7 MB/s\n'; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,259::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000557027 s, 3.7 MB/s'], size: 2048 >Thread-208::DEBUG::2014-10-23 17:16:19,259::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >Thread-208::DEBUG::2014-10-23 17:16:19,259::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >Thread-208::DEBUG::2014-10-23 17:16:19,259::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >Thread-208::DEBUG::2014-10-23 17:16:19,259::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,260::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,260::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,260::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,260::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,260::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,353::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,354::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,354::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-208::INFO::2014-10-23 17:16:19,355::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >Thread-208::INFO::2014-10-23 17:16:19,355::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >Thread-208::INFO::2014-10-23 17:16:19,355::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >Thread-208::DEBUG::2014-10-23 17:16:19,355::sp::1390::Storage.StoragePool::(setMasterDomain) Master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f verified, version 1 >Thread-208::DEBUG::2014-10-23 17:16:19,355::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-208::DEBUG::2014-10-23 17:16:19,355::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-208::INFO::2014-10-23 17:16:19,355::domainMonitor::100::Storage.DomainMonitor::(startMonitoring) Start monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-208::DEBUG::2014-10-23 17:16:19,356::sp::1413::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` started monitoring domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >Thread-209::DEBUG::2014-10-23 17:16:19,356::domainMonitor::182::Storage.DomainMonitorThread::(_monitorLoop) Starting domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-208::DEBUG::2014-10-23 17:16:19,356::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-208::DEBUG::2014-10-23 17:16:19,356::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-209::DEBUG::2014-10-23 17:16:19,357::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgck --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,362::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,362::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name (cwd None) >Thread-209::DEBUG::2014-10-23 17:16:19,452::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-209::DEBUG::2014-10-23 17:16:19,453::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-209::DEBUG::2014-10-23 17:16:19,458::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000397301 s, 10.3 MB/s\n'; <rc> = 0 >Thread-209::DEBUG::2014-10-23 17:16:19,461::domainMonitor::264::Storage.DomainMonitorThread::(_monitorDomain) Domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f changed its status to Valid >Thread-209::INFO::2014-10-23 17:16:19,462::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >Thread-210::DEBUG::2014-10-23 17:16:19,462::misc::821::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Emitting event >Thread-209::DEBUG::2014-10-23 17:16:19,463::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >Thread-210::DEBUG::2014-10-23 17:16:19,463::misc::831::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Calling registered method `contEIOVms` >Thread-210::DEBUG::2014-10-23 17:16:19,464::misc::841::Storage.Event.Storage.DomainMonitor.onDomainStateChange::(_emit) Event emitted >Thread-211::INFO::2014-10-23 17:16:19,470::clientIF::136::vds::(contEIOVms) vmContainerLock acquired >Thread-208::DEBUG::2014-10-23 17:16:19,470::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,471::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-208::DEBUG::2014-10-23 17:16:19,471::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >Thread-208::WARNING::2014-10-23 17:16:19,471::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >Thread-208::DEBUG::2014-10-23 17:16:19,471::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >Thread-208::WARNING::2014-10-23 17:16:19,471::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >Thread-208::INFO::2014-10-23 17:16:19,471::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-208::DEBUG::2014-10-23 17:16:19,472::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-208::INFO::2014-10-23 17:16:19,472::sp::1120::Storage.StoragePool::(_linkStorageDomain) Linking /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-208::DEBUG::2014-10-23 17:16:19,472::sp::1137::Storage.StoragePool::(_linkStorageDomain) Creating symlink from /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f to /rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd >Thread-208::DEBUG::2014-10-23 17:16:19,472::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >Thread-208::DEBUG::2014-10-23 17:16:19,474::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox iflag=direct,fullblock bs=512 count=8 skip=8 (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,481::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00285284 s, 1.4 MB/s\n'; <rc> = 0 >Thread-208::INFO::2014-10-23 17:16:19,481::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-208::DEBUG::2014-10-23 17:16:19,481::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-208::DEBUG::2014-10-23 17:16:19,488::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00257844 s, 1.6 MB/s\n'; <rc> = 0 >Thread-208::DEBUG::2014-10-23 17:16:19,488::storage_mailbox::207::Storage.Mailbox.HSM::(__init__) HSM_MailboxMonitor created for pool 00000002-0002-0002-0002-000000000067 >Thread-208::DEBUG::2014-10-23 17:16:19,489::sp::462::Storage.StoragePool::(__createMailboxMonitor) HSM mailbox ready for pool 00000002-0002-0002-0002-000000000067 on master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-208::DEBUG::2014-10-23 17:16:19,489::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-208::DEBUG::2014-10-23 17:16:19,490::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-208::INFO::2014-10-23 17:16:19,490::logUtils::47::dispatcher::(wrapper) Run and protect: connectStoragePool, Return response: True >Thread-208::DEBUG::2014-10-23 17:16:19,490::task::1191::Storage.TaskManager.Task::(prepare) Task=`3979b6ec-8288-49f1-b7b5-d65861623372`::finished: True >Thread-208::DEBUG::2014-10-23 17:16:19,490::task::595::Storage.TaskManager.Task::(_updateState) Task=`3979b6ec-8288-49f1-b7b5-d65861623372`::moving from state preparing -> state finished >Thread-208::DEBUG::2014-10-23 17:16:19,490::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-208::DEBUG::2014-10-23 17:16:19,490::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-208::DEBUG::2014-10-23 17:16:19,490::task::993::Storage.TaskManager.Task::(_decref) Task=`3979b6ec-8288-49f1-b7b5-d65861623372`::ref 0 aborting False >Thread-208::DEBUG::2014-10-23 17:16:19,490::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.connect' in bridge with True >Thread-208::DEBUG::2014-10-23 17:16:19,490::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:19,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:19,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-218::DEBUG::2014-10-23 17:16:19,572::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >Thread-218::DEBUG::2014-10-23 17:16:19,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e6edd07-317a-46a8-8146-8e2a4ecc6a89`::moving from state init -> state preparing >Thread-218::INFO::2014-10-23 17:16:19,573::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-218::INFO::2014-10-23 17:16:19,584::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-218::DEBUG::2014-10-23 17:16:19,584::task::1191::Storage.TaskManager.Task::(prepare) Task=`5e6edd07-317a-46a8-8146-8e2a4ecc6a89`::finished: {'spm_st': {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1}} >Thread-218::DEBUG::2014-10-23 17:16:19,585::task::595::Storage.TaskManager.Task::(_updateState) Task=`5e6edd07-317a-46a8-8146-8e2a4ecc6a89`::moving from state preparing -> state finished >Thread-218::DEBUG::2014-10-23 17:16:19,585::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-218::DEBUG::2014-10-23 17:16:19,585::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-218::DEBUG::2014-10-23 17:16:19,585::task::993::Storage.TaskManager.Task::(_decref) Task=`5e6edd07-317a-46a8-8146-8e2a4ecc6a89`::ref 0 aborting False >Thread-218::DEBUG::2014-10-23 17:16:19,585::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': -1, 'spmStatus': 'Free', 'spmLver': -1} >Thread-218::DEBUG::2014-10-23 17:16:19,585::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:19,614::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:19,614::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-219::DEBUG::2014-10-23 17:16:19,614::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.spmStart' in bridge with {u'maxHostID': 250, u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'prevLver': u'-1', u'enableScsiFencing': u'false', u'prevID': -1} >Thread-219::DEBUG::2014-10-23 17:16:19,615::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::moving from state init -> state preparing >Thread-219::INFO::2014-10-23 17:16:19,615::logUtils::44::dispatcher::(wrapper) Run and protect: spmStart(spUUID=u'00000002-0002-0002-0002-000000000067', prevID=-1, prevLVER=u'-1', maxHostID=250, domVersion=None, options=None) >Thread-219::DEBUG::2014-10-23 17:16:19,616::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`1291246e-3c3b-42fa-8921-89b91e8880b3`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '572' at 'spmStart' >Thread-219::DEBUG::2014-10-23 17:16:19,616::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-219::DEBUG::2014-10-23 17:16:19,616::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-219::DEBUG::2014-10-23 17:16:19,616::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`1291246e-3c3b-42fa-8921-89b91e8880b3`::Granted request >Thread-219::DEBUG::2014-10-23 17:16:19,616::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-219::DEBUG::2014-10-23 17:16:19,616::task::993::Storage.TaskManager.Task::(_decref) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::ref 1 aborting False >Thread-219::DEBUG::2014-10-23 17:16:19,617::taskManager::78::Storage.TaskManager::(scheduleJob) scheduled job spmStart for task f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-219::INFO::2014-10-23 17:16:19,617::logUtils::47::dispatcher::(wrapper) Run and protect: spmStart, Return response: None >Thread-219::DEBUG::2014-10-23 17:16:19,617::task::1183::Storage.TaskManager.Task::(prepare) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::Prepare: 1 jobs exist, move to acquiring >Thread-219::DEBUG::2014-10-23 17:16:19,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::moving from state preparing -> state acquiring >Thread-219::DEBUG::2014-10-23 17:16:19,617::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::moving from state acquiring -> state queued >Thread-219::DEBUG::2014-10-23 17:16:19,617::taskManager::56::Storage.TaskManager::(_queueTask) queuing task: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-219::DEBUG::2014-10-23 17:16:19,617::taskManager::64::Storage.TaskManager::(_queueTask) task queued: f2653993-1637-420a-9dfc-afcb1584c7ed >1c557727-9394-4848-9699-e6a94a5708d6::DEBUG::2014-10-23 17:16:19,617::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 1 >Thread-219::DEBUG::2014-10-23 17:16:19,618::task::1188::Storage.TaskManager.Task::(prepare) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::returning >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,618::threadPool::200::Storage.ThreadPool.WorkerThread::(_processNextTask) Task: f2653993-1637-420a-9dfc-afcb1584c7ed running: <bound method Task.commit of <storage.task.Task instance at 0x7fa0502243f8>> with: None >Thread-219::DEBUG::2014-10-23 17:16:19,618::task::993::Storage.TaskManager.Task::(_decref) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::ref 0 aborting False >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,618::task::1200::Storage.TaskManager.Task::(commit) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::committing task: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-219::DEBUG::2014-10-23 17:16:19,619::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.spmStart' in bridge with f2653993-1637-420a-9dfc-afcb1584c7ed >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,619::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::moving from state queued -> state running >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,619::task::905::Storage.TaskManager.Task::(_runJobs) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::Task.run: running job 0: spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fa03405efd0>> (args: (-1, u'-1', 250, None) kwargs: {}) >Thread-219::DEBUG::2014-10-23 17:16:19,619::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,620::task::331::Storage.TaskManager.Task::(run) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::Job.run: running spmStart: <bound method StoragePool.startSpm of <storage.sp.StoragePool object at 0x7fa03405efd0>> (args: (-1, u'-1', 250, None) kwargs: {}) callback None >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,620::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,621::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:19,621::misc::751::Storage.SamplingMethod::(__call__) Returning last result >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:19,632::clusterlock::222::Storage.SANLock::(acquireHostId) Acquiring host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:20,633::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:20,634::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-220::DEBUG::2014-10-23 17:16:20,634::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-220::DEBUG::2014-10-23 17:16:20,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`35c68bef-8fa4-4f66-81aa-f9c0a348b55d`::moving from state init -> state preparing >Thread-220::INFO::2014-10-23 17:16:20,635::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-220::DEBUG::2014-10-23 17:16:20,635::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-220::DEBUG::2014-10-23 17:16:20,635::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-220::INFO::2014-10-23 17:16:20,635::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-220::DEBUG::2014-10-23 17:16:20,635::task::1191::Storage.TaskManager.Task::(prepare) Task=`35c68bef-8fa4-4f66-81aa-f9c0a348b55d`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-220::DEBUG::2014-10-23 17:16:20,635::task::595::Storage.TaskManager.Task::(_updateState) Task=`35c68bef-8fa4-4f66-81aa-f9c0a348b55d`::moving from state preparing -> state finished >Thread-220::DEBUG::2014-10-23 17:16:20,635::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-220::DEBUG::2014-10-23 17:16:20,635::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-220::DEBUG::2014-10-23 17:16:20,636::task::993::Storage.TaskManager.Task::(_decref) Task=`35c68bef-8fa4-4f66-81aa-f9c0a348b55d`::ref 0 aborting False >Thread-220::DEBUG::2014-10-23 17:16:20,636::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-220::DEBUG::2014-10-23 17:16:20,636::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:20,888::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:20,889::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-221::DEBUG::2014-10-23 17:16:20,890::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:21,647::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-222::DEBUG::2014-10-23 17:16:21,647::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:21,647::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-222::DEBUG::2014-10-23 17:16:21,648::task::595::Storage.TaskManager.Task::(_updateState) Task=`542ab25f-a0e4-4368-befb-0b0cd1259da9`::moving from state init -> state preparing >Thread-222::INFO::2014-10-23 17:16:21,648::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-222::DEBUG::2014-10-23 17:16:21,648::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-222::DEBUG::2014-10-23 17:16:21,649::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-222::INFO::2014-10-23 17:16:21,649::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-222::DEBUG::2014-10-23 17:16:21,649::task::1191::Storage.TaskManager.Task::(prepare) Task=`542ab25f-a0e4-4368-befb-0b0cd1259da9`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-222::DEBUG::2014-10-23 17:16:21,649::task::595::Storage.TaskManager.Task::(_updateState) Task=`542ab25f-a0e4-4368-befb-0b0cd1259da9`::moving from state preparing -> state finished >Thread-222::DEBUG::2014-10-23 17:16:21,649::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-222::DEBUG::2014-10-23 17:16:21,649::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-222::DEBUG::2014-10-23 17:16:21,649::task::993::Storage.TaskManager.Task::(_decref) Task=`542ab25f-a0e4-4368-befb-0b0cd1259da9`::ref 0 aborting False >Thread-222::DEBUG::2014-10-23 17:16:21,649::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-222::DEBUG::2014-10-23 17:16:21,649::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:22,652::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:22,652::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-223::DEBUG::2014-10-23 17:16:22,652::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-223::DEBUG::2014-10-23 17:16:22,653::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb84760b-129b-485e-8eb1-d535db7c3fd3`::moving from state init -> state preparing >Thread-223::INFO::2014-10-23 17:16:22,653::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-223::DEBUG::2014-10-23 17:16:22,653::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-223::DEBUG::2014-10-23 17:16:22,653::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-223::INFO::2014-10-23 17:16:22,653::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-223::DEBUG::2014-10-23 17:16:22,654::task::1191::Storage.TaskManager.Task::(prepare) Task=`fb84760b-129b-485e-8eb1-d535db7c3fd3`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-223::DEBUG::2014-10-23 17:16:22,654::task::595::Storage.TaskManager.Task::(_updateState) Task=`fb84760b-129b-485e-8eb1-d535db7c3fd3`::moving from state preparing -> state finished >Thread-223::DEBUG::2014-10-23 17:16:22,654::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-223::DEBUG::2014-10-23 17:16:22,654::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-223::DEBUG::2014-10-23 17:16:22,654::task::993::Storage.TaskManager.Task::(_decref) Task=`fb84760b-129b-485e-8eb1-d535db7c3fd3`::ref 0 aborting False >Thread-223::DEBUG::2014-10-23 17:16:22,654::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-223::DEBUG::2014-10-23 17:16:22,654::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:23,657::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:23,658::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-224::DEBUG::2014-10-23 17:16:23,658::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-224::DEBUG::2014-10-23 17:16:23,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b36e1cc-a55b-48f4-b4e6-6befa8bd1dd2`::moving from state init -> state preparing >Thread-224::INFO::2014-10-23 17:16:23,659::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-224::DEBUG::2014-10-23 17:16:23,659::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-224::DEBUG::2014-10-23 17:16:23,659::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-224::INFO::2014-10-23 17:16:23,659::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-224::DEBUG::2014-10-23 17:16:23,659::task::1191::Storage.TaskManager.Task::(prepare) Task=`4b36e1cc-a55b-48f4-b4e6-6befa8bd1dd2`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-224::DEBUG::2014-10-23 17:16:23,659::task::595::Storage.TaskManager.Task::(_updateState) Task=`4b36e1cc-a55b-48f4-b4e6-6befa8bd1dd2`::moving from state preparing -> state finished >Thread-224::DEBUG::2014-10-23 17:16:23,659::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-224::DEBUG::2014-10-23 17:16:23,659::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-224::DEBUG::2014-10-23 17:16:23,660::task::993::Storage.TaskManager.Task::(_decref) Task=`4b36e1cc-a55b-48f4-b4e6-6befa8bd1dd2`::ref 0 aborting False >Thread-224::DEBUG::2014-10-23 17:16:23,660::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-224::DEBUG::2014-10-23 17:16:23,660::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:23,904::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:23,905::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-225::DEBUG::2014-10-23 17:16:23,906::task::595::Storage.TaskManager.Task::(_updateState) Task=`4194cd31-d888-4cd2-8546-6d31e3a7bb9e`::moving from state init -> state preparing >Thread-225::INFO::2014-10-23 17:16:23,906::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-225::INFO::2014-10-23 17:16:23,906::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000397301', 'lastCheck': '4.4', 'valid': True}} >Thread-225::DEBUG::2014-10-23 17:16:23,906::task::1191::Storage.TaskManager.Task::(prepare) Task=`4194cd31-d888-4cd2-8546-6d31e3a7bb9e`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000397301', 'lastCheck': '4.4', 'valid': True}} >Thread-225::DEBUG::2014-10-23 17:16:23,907::task::595::Storage.TaskManager.Task::(_updateState) Task=`4194cd31-d888-4cd2-8546-6d31e3a7bb9e`::moving from state preparing -> state finished >Thread-225::DEBUG::2014-10-23 17:16:23,907::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-225::DEBUG::2014-10-23 17:16:23,907::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-225::DEBUG::2014-10-23 17:16:23,907::task::993::Storage.TaskManager.Task::(_decref) Task=`4194cd31-d888-4cd2-8546-6d31e3a7bb9e`::ref 0 aborting False >Thread-225::DEBUG::2014-10-23 17:16:23,908::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:23,938::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:23,938::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-226::DEBUG::2014-10-23 17:16:23,939::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:24,672::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:24,673::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-227::DEBUG::2014-10-23 17:16:24,673::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-227::DEBUG::2014-10-23 17:16:24,674::task::595::Storage.TaskManager.Task::(_updateState) Task=`86c87e17-e659-45a9-89dc-b63778b766fe`::moving from state init -> state preparing >Thread-227::INFO::2014-10-23 17:16:24,674::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-227::DEBUG::2014-10-23 17:16:24,674::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-227::DEBUG::2014-10-23 17:16:24,675::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-227::INFO::2014-10-23 17:16:24,675::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-227::DEBUG::2014-10-23 17:16:24,675::task::1191::Storage.TaskManager.Task::(prepare) Task=`86c87e17-e659-45a9-89dc-b63778b766fe`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-227::DEBUG::2014-10-23 17:16:24,675::task::595::Storage.TaskManager.Task::(_updateState) Task=`86c87e17-e659-45a9-89dc-b63778b766fe`::moving from state preparing -> state finished >Thread-227::DEBUG::2014-10-23 17:16:24,675::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-227::DEBUG::2014-10-23 17:16:24,675::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-227::DEBUG::2014-10-23 17:16:24,675::task::993::Storage.TaskManager.Task::(_decref) Task=`86c87e17-e659-45a9-89dc-b63778b766fe`::ref 0 aborting False >Thread-227::DEBUG::2014-10-23 17:16:24,675::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-227::DEBUG::2014-10-23 17:16:24,675::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:25,678::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:25,678::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-228::DEBUG::2014-10-23 17:16:25,678::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-228::DEBUG::2014-10-23 17:16:25,679::task::595::Storage.TaskManager.Task::(_updateState) Task=`56e618b1-4b28-4785-9566-f81bc2a0c697`::moving from state init -> state preparing >Thread-228::INFO::2014-10-23 17:16:25,679::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-228::DEBUG::2014-10-23 17:16:25,679::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-228::DEBUG::2014-10-23 17:16:25,679::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-228::INFO::2014-10-23 17:16:25,679::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-228::DEBUG::2014-10-23 17:16:25,679::task::1191::Storage.TaskManager.Task::(prepare) Task=`56e618b1-4b28-4785-9566-f81bc2a0c697`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-228::DEBUG::2014-10-23 17:16:25,680::task::595::Storage.TaskManager.Task::(_updateState) Task=`56e618b1-4b28-4785-9566-f81bc2a0c697`::moving from state preparing -> state finished >Thread-228::DEBUG::2014-10-23 17:16:25,680::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-228::DEBUG::2014-10-23 17:16:25,680::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-228::DEBUG::2014-10-23 17:16:25,680::task::993::Storage.TaskManager.Task::(_decref) Task=`56e618b1-4b28-4785-9566-f81bc2a0c697`::ref 0 aborting False >Thread-228::DEBUG::2014-10-23 17:16:25,680::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-228::DEBUG::2014-10-23 17:16:25,680::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:26,685::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:26,685::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-229::DEBUG::2014-10-23 17:16:26,685::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-229::DEBUG::2014-10-23 17:16:26,686::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b3646a4-60b5-4c54-aa4b-bd2a8e317b1f`::moving from state init -> state preparing >Thread-229::INFO::2014-10-23 17:16:26,686::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-229::DEBUG::2014-10-23 17:16:26,686::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-229::DEBUG::2014-10-23 17:16:26,687::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-229::INFO::2014-10-23 17:16:26,687::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-229::DEBUG::2014-10-23 17:16:26,687::task::1191::Storage.TaskManager.Task::(prepare) Task=`7b3646a4-60b5-4c54-aa4b-bd2a8e317b1f`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-229::DEBUG::2014-10-23 17:16:26,687::task::595::Storage.TaskManager.Task::(_updateState) Task=`7b3646a4-60b5-4c54-aa4b-bd2a8e317b1f`::moving from state preparing -> state finished >Thread-229::DEBUG::2014-10-23 17:16:26,687::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-229::DEBUG::2014-10-23 17:16:26,687::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-229::DEBUG::2014-10-23 17:16:26,687::task::993::Storage.TaskManager.Task::(_decref) Task=`7b3646a4-60b5-4c54-aa4b-bd2a8e317b1f`::ref 0 aborting False >Thread-229::DEBUG::2014-10-23 17:16:26,687::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-229::DEBUG::2014-10-23 17:16:26,687::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:27,110::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:27,111::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-230::DEBUG::2014-10-23 17:16:27,112::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:27,690::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-231::DEBUG::2014-10-23 17:16:27,690::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:27,690::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-231::DEBUG::2014-10-23 17:16:27,691::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d7941b7-e620-4ad6-957e-4123a940fe2b`::moving from state init -> state preparing >Thread-231::INFO::2014-10-23 17:16:27,691::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-231::DEBUG::2014-10-23 17:16:27,691::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-231::DEBUG::2014-10-23 17:16:27,691::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-231::INFO::2014-10-23 17:16:27,691::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-231::DEBUG::2014-10-23 17:16:27,692::task::1191::Storage.TaskManager.Task::(prepare) Task=`4d7941b7-e620-4ad6-957e-4123a940fe2b`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-231::DEBUG::2014-10-23 17:16:27,692::task::595::Storage.TaskManager.Task::(_updateState) Task=`4d7941b7-e620-4ad6-957e-4123a940fe2b`::moving from state preparing -> state finished >Thread-231::DEBUG::2014-10-23 17:16:27,692::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-231::DEBUG::2014-10-23 17:16:27,692::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-231::DEBUG::2014-10-23 17:16:27,692::task::993::Storage.TaskManager.Task::(_decref) Task=`4d7941b7-e620-4ad6-957e-4123a940fe2b`::ref 0 aborting False >Thread-231::DEBUG::2014-10-23 17:16:27,692::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-231::DEBUG::2014-10-23 17:16:27,692::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:28,695::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:28,696::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-232::DEBUG::2014-10-23 17:16:28,696::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-232::DEBUG::2014-10-23 17:16:28,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dc98bb4-da09-43de-811f-1e2e1f22e3e3`::moving from state init -> state preparing >Thread-232::INFO::2014-10-23 17:16:28,697::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-232::DEBUG::2014-10-23 17:16:28,697::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-232::DEBUG::2014-10-23 17:16:28,697::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-232::INFO::2014-10-23 17:16:28,697::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-232::DEBUG::2014-10-23 17:16:28,697::task::1191::Storage.TaskManager.Task::(prepare) Task=`7dc98bb4-da09-43de-811f-1e2e1f22e3e3`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-232::DEBUG::2014-10-23 17:16:28,697::task::595::Storage.TaskManager.Task::(_updateState) Task=`7dc98bb4-da09-43de-811f-1e2e1f22e3e3`::moving from state preparing -> state finished >Thread-232::DEBUG::2014-10-23 17:16:28,698::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-232::DEBUG::2014-10-23 17:16:28,698::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-232::DEBUG::2014-10-23 17:16:28,698::task::993::Storage.TaskManager.Task::(_decref) Task=`7dc98bb4-da09-43de-811f-1e2e1f22e3e3`::ref 0 aborting False >Thread-232::DEBUG::2014-10-23 17:16:28,698::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-232::DEBUG::2014-10-23 17:16:28,698::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-209::DEBUG::2014-10-23 17:16:29,464::blockSD::596::Storage.Misc.excCmd::(getReadDelay) /bin/dd if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata iflag=direct of=/dev/null bs=4096 count=1 (cwd None) >Thread-209::DEBUG::2014-10-23 17:16:29,470::blockSD::596::Storage.Misc.excCmd::(getReadDelay) SUCCESS: <err> = '1+0 records in\n1+0 records out\n4096 bytes (4.1 kB) copied, 0.000560104 s, 7.3 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:29,700::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:29,701::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-233::DEBUG::2014-10-23 17:16:29,701::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-233::DEBUG::2014-10-23 17:16:29,702::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb26e134-fdf6-49fa-9069-a83090c72207`::moving from state init -> state preparing >Thread-233::INFO::2014-10-23 17:16:29,703::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-233::DEBUG::2014-10-23 17:16:29,703::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-233::DEBUG::2014-10-23 17:16:29,703::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-233::INFO::2014-10-23 17:16:29,703::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-233::DEBUG::2014-10-23 17:16:29,703::task::1191::Storage.TaskManager.Task::(prepare) Task=`cb26e134-fdf6-49fa-9069-a83090c72207`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-233::DEBUG::2014-10-23 17:16:29,703::task::595::Storage.TaskManager.Task::(_updateState) Task=`cb26e134-fdf6-49fa-9069-a83090c72207`::moving from state preparing -> state finished >Thread-233::DEBUG::2014-10-23 17:16:29,703::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-233::DEBUG::2014-10-23 17:16:29,703::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-233::DEBUG::2014-10-23 17:16:29,703::task::993::Storage.TaskManager.Task::(_decref) Task=`cb26e134-fdf6-49fa-9069-a83090c72207`::ref 0 aborting False >Thread-233::DEBUG::2014-10-23 17:16:29,703::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-233::DEBUG::2014-10-23 17:16:29,704::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:30,133::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:30,133::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-234::DEBUG::2014-10-23 17:16:30,134::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:30,706::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:30,707::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-235::DEBUG::2014-10-23 17:16:30,707::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-235::DEBUG::2014-10-23 17:16:30,708::task::595::Storage.TaskManager.Task::(_updateState) Task=`d135d9a8-c709-4069-8fc6-d16fb856af77`::moving from state init -> state preparing >Thread-235::INFO::2014-10-23 17:16:30,708::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-235::DEBUG::2014-10-23 17:16:30,708::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-235::DEBUG::2014-10-23 17:16:30,708::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-235::INFO::2014-10-23 17:16:30,708::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-235::DEBUG::2014-10-23 17:16:30,708::task::1191::Storage.TaskManager.Task::(prepare) Task=`d135d9a8-c709-4069-8fc6-d16fb856af77`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-235::DEBUG::2014-10-23 17:16:30,708::task::595::Storage.TaskManager.Task::(_updateState) Task=`d135d9a8-c709-4069-8fc6-d16fb856af77`::moving from state preparing -> state finished >Thread-235::DEBUG::2014-10-23 17:16:30,708::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-235::DEBUG::2014-10-23 17:16:30,708::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-235::DEBUG::2014-10-23 17:16:30,708::task::993::Storage.TaskManager.Task::(_decref) Task=`d135d9a8-c709-4069-8fc6-d16fb856af77`::ref 0 aborting False >Thread-235::DEBUG::2014-10-23 17:16:30,708::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-235::DEBUG::2014-10-23 17:16:30,709::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:31,715::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:31,715::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-236::DEBUG::2014-10-23 17:16:31,715::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-236::DEBUG::2014-10-23 17:16:31,716::task::595::Storage.TaskManager.Task::(_updateState) Task=`82c3baa8-861f-43e7-ad55-da1f2526b866`::moving from state init -> state preparing >Thread-236::INFO::2014-10-23 17:16:31,716::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-236::DEBUG::2014-10-23 17:16:31,716::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-236::DEBUG::2014-10-23 17:16:31,716::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-236::INFO::2014-10-23 17:16:31,717::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-236::DEBUG::2014-10-23 17:16:31,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`82c3baa8-861f-43e7-ad55-da1f2526b866`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-236::DEBUG::2014-10-23 17:16:31,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`82c3baa8-861f-43e7-ad55-da1f2526b866`::moving from state preparing -> state finished >Thread-236::DEBUG::2014-10-23 17:16:31,717::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-236::DEBUG::2014-10-23 17:16:31,717::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-236::DEBUG::2014-10-23 17:16:31,717::task::993::Storage.TaskManager.Task::(_decref) Task=`82c3baa8-861f-43e7-ad55-da1f2526b866`::ref 0 aborting False >Thread-236::DEBUG::2014-10-23 17:16:31,717::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-236::DEBUG::2014-10-23 17:16:31,717::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:32,720::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:32,720::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-237::DEBUG::2014-10-23 17:16:32,720::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-237::DEBUG::2014-10-23 17:16:32,721::task::595::Storage.TaskManager.Task::(_updateState) Task=`20449035-5de2-4294-91f4-e2c9c8b8a02d`::moving from state init -> state preparing >Thread-237::INFO::2014-10-23 17:16:32,721::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-237::DEBUG::2014-10-23 17:16:32,721::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-237::DEBUG::2014-10-23 17:16:32,721::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-237::INFO::2014-10-23 17:16:32,721::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-237::DEBUG::2014-10-23 17:16:32,721::task::1191::Storage.TaskManager.Task::(prepare) Task=`20449035-5de2-4294-91f4-e2c9c8b8a02d`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-237::DEBUG::2014-10-23 17:16:32,722::task::595::Storage.TaskManager.Task::(_updateState) Task=`20449035-5de2-4294-91f4-e2c9c8b8a02d`::moving from state preparing -> state finished >Thread-237::DEBUG::2014-10-23 17:16:32,722::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-237::DEBUG::2014-10-23 17:16:32,722::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-237::DEBUG::2014-10-23 17:16:32,722::task::993::Storage.TaskManager.Task::(_decref) Task=`20449035-5de2-4294-91f4-e2c9c8b8a02d`::ref 0 aborting False >Thread-237::DEBUG::2014-10-23 17:16:32,722::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-237::DEBUG::2014-10-23 17:16:32,722::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:33,147::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:33,148::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-238::DEBUG::2014-10-23 17:16:33,149::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:33,728::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:33,728::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-239::DEBUG::2014-10-23 17:16:33,728::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-239::DEBUG::2014-10-23 17:16:33,729::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc36e1af-7210-46df-b167-10b6a673c996`::moving from state init -> state preparing >Thread-239::INFO::2014-10-23 17:16:33,729::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-239::DEBUG::2014-10-23 17:16:33,729::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-239::DEBUG::2014-10-23 17:16:33,730::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-239::INFO::2014-10-23 17:16:33,730::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-239::DEBUG::2014-10-23 17:16:33,730::task::1191::Storage.TaskManager.Task::(prepare) Task=`bc36e1af-7210-46df-b167-10b6a673c996`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-239::DEBUG::2014-10-23 17:16:33,730::task::595::Storage.TaskManager.Task::(_updateState) Task=`bc36e1af-7210-46df-b167-10b6a673c996`::moving from state preparing -> state finished >Thread-239::DEBUG::2014-10-23 17:16:33,730::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-239::DEBUG::2014-10-23 17:16:33,730::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-239::DEBUG::2014-10-23 17:16:33,730::task::993::Storage.TaskManager.Task::(_decref) Task=`bc36e1af-7210-46df-b167-10b6a673c996`::ref 0 aborting False >Thread-239::DEBUG::2014-10-23 17:16:33,730::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-239::DEBUG::2014-10-23 17:16:33,730::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:34,733::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-240::DEBUG::2014-10-23 17:16:34,734::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:34,734::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-240::DEBUG::2014-10-23 17:16:34,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca3bf734-0179-4c68-afd3-aba78c7e4a86`::moving from state init -> state preparing >Thread-240::INFO::2014-10-23 17:16:34,735::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-240::DEBUG::2014-10-23 17:16:34,735::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-240::DEBUG::2014-10-23 17:16:34,735::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-240::INFO::2014-10-23 17:16:34,735::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-240::DEBUG::2014-10-23 17:16:34,735::task::1191::Storage.TaskManager.Task::(prepare) Task=`ca3bf734-0179-4c68-afd3-aba78c7e4a86`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-240::DEBUG::2014-10-23 17:16:34,735::task::595::Storage.TaskManager.Task::(_updateState) Task=`ca3bf734-0179-4c68-afd3-aba78c7e4a86`::moving from state preparing -> state finished >Thread-240::DEBUG::2014-10-23 17:16:34,735::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-240::DEBUG::2014-10-23 17:16:34,735::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-240::DEBUG::2014-10-23 17:16:34,735::task::993::Storage.TaskManager.Task::(_decref) Task=`ca3bf734-0179-4c68-afd3-aba78c7e4a86`::ref 0 aborting False >Thread-240::DEBUG::2014-10-23 17:16:34,735::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-240::DEBUG::2014-10-23 17:16:34,736::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:35,738::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-241::DEBUG::2014-10-23 17:16:35,738::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:35,738::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-241::DEBUG::2014-10-23 17:16:35,739::task::595::Storage.TaskManager.Task::(_updateState) Task=`e80cbd52-ea87-4a80-b863-7e4d1a0524ce`::moving from state init -> state preparing >Thread-241::INFO::2014-10-23 17:16:35,739::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-241::DEBUG::2014-10-23 17:16:35,740::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-241::DEBUG::2014-10-23 17:16:35,740::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-241::INFO::2014-10-23 17:16:35,740::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-241::DEBUG::2014-10-23 17:16:35,740::task::1191::Storage.TaskManager.Task::(prepare) Task=`e80cbd52-ea87-4a80-b863-7e4d1a0524ce`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-241::DEBUG::2014-10-23 17:16:35,740::task::595::Storage.TaskManager.Task::(_updateState) Task=`e80cbd52-ea87-4a80-b863-7e4d1a0524ce`::moving from state preparing -> state finished >Thread-241::DEBUG::2014-10-23 17:16:35,740::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-241::DEBUG::2014-10-23 17:16:35,740::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-241::DEBUG::2014-10-23 17:16:35,740::task::993::Storage.TaskManager.Task::(_decref) Task=`e80cbd52-ea87-4a80-b863-7e4d1a0524ce`::ref 0 aborting False >Thread-241::DEBUG::2014-10-23 17:16:35,740::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-241::DEBUG::2014-10-23 17:16:35,740::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:36,161::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:36,162::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-242::DEBUG::2014-10-23 17:16:36,163::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:36,743::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:36,743::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-243::DEBUG::2014-10-23 17:16:36,743::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-243::DEBUG::2014-10-23 17:16:36,744::task::595::Storage.TaskManager.Task::(_updateState) Task=`24c9f492-fcc3-4cdd-b431-d0742ffdd7fc`::moving from state init -> state preparing >Thread-243::INFO::2014-10-23 17:16:36,744::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-243::DEBUG::2014-10-23 17:16:36,744::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-243::DEBUG::2014-10-23 17:16:36,744::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-243::INFO::2014-10-23 17:16:36,744::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-243::DEBUG::2014-10-23 17:16:36,744::task::1191::Storage.TaskManager.Task::(prepare) Task=`24c9f492-fcc3-4cdd-b431-d0742ffdd7fc`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-243::DEBUG::2014-10-23 17:16:36,745::task::595::Storage.TaskManager.Task::(_updateState) Task=`24c9f492-fcc3-4cdd-b431-d0742ffdd7fc`::moving from state preparing -> state finished >Thread-243::DEBUG::2014-10-23 17:16:36,745::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-243::DEBUG::2014-10-23 17:16:36,745::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-243::DEBUG::2014-10-23 17:16:36,745::task::993::Storage.TaskManager.Task::(_decref) Task=`24c9f492-fcc3-4cdd-b431-d0742ffdd7fc`::ref 0 aborting False >Thread-243::DEBUG::2014-10-23 17:16:36,745::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-243::DEBUG::2014-10-23 17:16:36,745::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:37,747::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:37,748::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-244::DEBUG::2014-10-23 17:16:37,748::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-244::DEBUG::2014-10-23 17:16:37,749::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d142b7d-e698-4a7f-a55a-b391f862a0b1`::moving from state init -> state preparing >Thread-244::INFO::2014-10-23 17:16:37,749::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-244::DEBUG::2014-10-23 17:16:37,749::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-244::DEBUG::2014-10-23 17:16:37,749::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-244::INFO::2014-10-23 17:16:37,749::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-244::DEBUG::2014-10-23 17:16:37,749::task::1191::Storage.TaskManager.Task::(prepare) Task=`3d142b7d-e698-4a7f-a55a-b391f862a0b1`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-244::DEBUG::2014-10-23 17:16:37,749::task::595::Storage.TaskManager.Task::(_updateState) Task=`3d142b7d-e698-4a7f-a55a-b391f862a0b1`::moving from state preparing -> state finished >Thread-244::DEBUG::2014-10-23 17:16:37,749::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-244::DEBUG::2014-10-23 17:16:37,749::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-244::DEBUG::2014-10-23 17:16:37,750::task::993::Storage.TaskManager.Task::(_decref) Task=`3d142b7d-e698-4a7f-a55a-b391f862a0b1`::ref 0 aborting False >Thread-244::DEBUG::2014-10-23 17:16:37,750::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-244::DEBUG::2014-10-23 17:16:37,750::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:38,752::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-245::DEBUG::2014-10-23 17:16:38,753::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:38,753::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-245::DEBUG::2014-10-23 17:16:38,754::task::595::Storage.TaskManager.Task::(_updateState) Task=`580aec9e-5bba-4bc0-8444-34438e43be0c`::moving from state init -> state preparing >Thread-245::INFO::2014-10-23 17:16:38,754::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-245::DEBUG::2014-10-23 17:16:38,754::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-245::DEBUG::2014-10-23 17:16:38,754::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-245::INFO::2014-10-23 17:16:38,755::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-245::DEBUG::2014-10-23 17:16:38,755::task::1191::Storage.TaskManager.Task::(prepare) Task=`580aec9e-5bba-4bc0-8444-34438e43be0c`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-245::DEBUG::2014-10-23 17:16:38,755::task::595::Storage.TaskManager.Task::(_updateState) Task=`580aec9e-5bba-4bc0-8444-34438e43be0c`::moving from state preparing -> state finished >Thread-245::DEBUG::2014-10-23 17:16:38,755::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-245::DEBUG::2014-10-23 17:16:38,755::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-245::DEBUG::2014-10-23 17:16:38,755::task::993::Storage.TaskManager.Task::(_decref) Task=`580aec9e-5bba-4bc0-8444-34438e43be0c`::ref 0 aborting False >Thread-245::DEBUG::2014-10-23 17:16:38,755::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-245::DEBUG::2014-10-23 17:16:38,755::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:39,176::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:39,177::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-246::DEBUG::2014-10-23 17:16:39,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dbefa3f-2041-44cc-a544-8abf527f284a`::moving from state init -> state preparing >Thread-246::INFO::2014-10-23 17:16:39,178::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-246::INFO::2014-10-23 17:16:39,178::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000397301', 'lastCheck': '19.7', 'valid': True}} >Thread-246::DEBUG::2014-10-23 17:16:39,178::task::1191::Storage.TaskManager.Task::(prepare) Task=`8dbefa3f-2041-44cc-a544-8abf527f284a`::finished: {u'5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': {'code': 0, 'version': 3, 'acquired': False, 'delay': '0.000397301', 'lastCheck': '19.7', 'valid': True}} >Thread-246::DEBUG::2014-10-23 17:16:39,178::task::595::Storage.TaskManager.Task::(_updateState) Task=`8dbefa3f-2041-44cc-a544-8abf527f284a`::moving from state preparing -> state finished >Thread-246::DEBUG::2014-10-23 17:16:39,178::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-246::DEBUG::2014-10-23 17:16:39,178::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-246::DEBUG::2014-10-23 17:16:39,179::task::993::Storage.TaskManager.Task::(_decref) Task=`8dbefa3f-2041-44cc-a544-8abf527f284a`::ref 0 aborting False >Thread-246::DEBUG::2014-10-23 17:16:39,180::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:39,216::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:39,216::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-247::DEBUG::2014-10-23 17:16:39,217::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:39,760::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:39,761::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-248::DEBUG::2014-10-23 17:16:39,761::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-248::DEBUG::2014-10-23 17:16:39,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`21b5269b-89c5-4a28-966d-ac847d8b9ae6`::moving from state init -> state preparing >Thread-248::INFO::2014-10-23 17:16:39,762::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-248::DEBUG::2014-10-23 17:16:39,762::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-248::DEBUG::2014-10-23 17:16:39,762::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-248::INFO::2014-10-23 17:16:39,762::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-248::DEBUG::2014-10-23 17:16:39,762::task::1191::Storage.TaskManager.Task::(prepare) Task=`21b5269b-89c5-4a28-966d-ac847d8b9ae6`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-248::DEBUG::2014-10-23 17:16:39,762::task::595::Storage.TaskManager.Task::(_updateState) Task=`21b5269b-89c5-4a28-966d-ac847d8b9ae6`::moving from state preparing -> state finished >Thread-248::DEBUG::2014-10-23 17:16:39,763::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-248::DEBUG::2014-10-23 17:16:39,763::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-248::DEBUG::2014-10-23 17:16:39,763::task::993::Storage.TaskManager.Task::(_decref) Task=`21b5269b-89c5-4a28-966d-ac847d8b9ae6`::ref 0 aborting False >Thread-248::DEBUG::2014-10-23 17:16:39,763::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-248::DEBUG::2014-10-23 17:16:39,763::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,633::clusterlock::240::Storage.SANLock::(acquireHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,633::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images >f2653993-1637-420a-9dfc-afcb1584c7ed::WARNING::2014-10-23 17:16:40,634::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/images already exists >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,634::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md >f2653993-1637-420a-9dfc-afcb1584c7ed::WARNING::2014-10-23 17:16:40,634::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/dom_md already exists >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,634::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,635::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,635::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,635::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,635::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,635::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,640::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000427008 s, 4.8 MB/s\n'; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,640::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000427008 s, 4.8 MB/s'], size: 2048 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,640::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,641::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,642::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,750::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,751::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,752::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:40,752::clusterlock::295::Storage.SANLock::(acquire) Acquiring cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:40,773::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:40,774::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-249::DEBUG::2014-10-23 17:16:40,774::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-249::DEBUG::2014-10-23 17:16:40,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`c903d2b8-07d8-4370-8192-56293cb7dd08`::moving from state init -> state preparing >Thread-249::INFO::2014-10-23 17:16:40,775::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-249::DEBUG::2014-10-23 17:16:40,775::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-249::DEBUG::2014-10-23 17:16:40,775::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-249::INFO::2014-10-23 17:16:40,775::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-249::DEBUG::2014-10-23 17:16:40,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`c903d2b8-07d8-4370-8192-56293cb7dd08`::finished: {'taskStatus': {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-249::DEBUG::2014-10-23 17:16:40,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`c903d2b8-07d8-4370-8192-56293cb7dd08`::moving from state preparing -> state finished >Thread-249::DEBUG::2014-10-23 17:16:40,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-249::DEBUG::2014-10-23 17:16:40,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-249::DEBUG::2014-10-23 17:16:40,776::task::993::Storage.TaskManager.Task::(_decref) Task=`c903d2b8-07d8-4370-8192-56293cb7dd08`::ref 0 aborting False >Thread-249::DEBUG::2014-10-23 17:16:40,776::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': 'running job 1 of 1', 'taskState': 'running', 'taskResult': '', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-249::DEBUG::2014-10-23 17:16:40,776::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,801::clusterlock::321::Storage.SANLock::(acquire) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully acquired (id: 1) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,801::sp::272::Storage.StoragePool::(startSpm) spm lock acquired successfully >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,801::spbackends::422::Storage.StoragePoolMemoryBackend::(setSpmStatus) this storage pool implementation ignores the set spm status requests (lver=0, spmid=1) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,801::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`f9de63ae-b9ed-4727-808d-48e310912e55`::Request was made in '/usr/share/vdsm/storage/sp.py' line '426' at '_upgradePool' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,802::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,802::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,802::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`f9de63ae-b9ed-4727-808d-48e310912e55`::Granted request >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:40,802::sp::432::Storage.StoragePool::(_upgradePool) Trying to upgrade master domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,803::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`75c94238-3684-4264-bf55-4ce2d04a7f86`::Request was made in '/usr/share/vdsm/storage/sp.py' line '434' at '_upgradePool' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,803::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,803::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,803::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`75c94238-3684-4264-bf55-4ce2d04a7f86`::Granted request >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,805::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with LvMetadataRW backend >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,805::blockSD::336::Storage.Misc.excCmd::(readlines) /bin/dd iflag=direct skip=0 bs=2048 if=/dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/metadata count=1 (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,810::blockSD::336::Storage.Misc.excCmd::(readlines) SUCCESS: <err> = '1+0 records in\n1+0 records out\n2048 bytes (2.0 kB) copied, 0.000470753 s, 4.4 MB/s\n'; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,810::misc::262::Storage.Misc::(validateDDBytes) err: ['1+0 records in', '1+0 records out', '2048 bytes (2.0 kB) copied, 0.000470753 s, 4.4 MB/s'], size: 2048 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,810::persistentDict::234::Storage.PersistentDict::(refresh) read lines (LvMetadataRW)=[] >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,810::persistentDict::252::Storage.PersistentDict::(refresh) Empty metadata >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::persistentDict::192::Storage.PersistentDict::(__init__) Created a persistent dict with VGTagMetadataRW backend >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,811::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,812::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,907::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,907::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,908::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:40,908::sd::385::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_imageNS already registered >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:40,908::sd::393::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_volumeNS already registered >f2653993-1637-420a-9dfc-afcb1584c7ed::INFO::2014-10-23 17:16:40,908::blockSD::448::Storage.StorageDomain::(_registerResourceNamespaces) Resource namespace 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f_lvmActivationNS already registered >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::sp::437::Storage.StoragePool::(_upgradePool) Marking all domains for upgrade >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::sp::444::Storage.StoragePool::(_upgradePool) Registering with state change event >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::sp::447::Storage.StoragePool::(_upgradePool) Running initial domain upgrade threads >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,909::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,910::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' (0 active users) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,910::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,910::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067', Clearing records. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,910::fileUtils::142::Storage.fileUtils::(createdir) Creating directory: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master >f2653993-1637-420a-9dfc-afcb1584c7ed::WARNING::2014-10-23 17:16:40,910::fileUtils::149::Storage.fileUtils::(createdir) Dir /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master already exists >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,910::blockSD::1126::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /sbin/fsck -p /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,949::blockSD::1126::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,949::blockSD::1151::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /sbin/tune2fs -j /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,959::blockSD::1151::Storage.Misc.excCmd::(mountMaster) FAILED: <err> = 'The filesystem already has a journal.\n'; <rc> = 1 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,959::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/mount -t ext3 /dev/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,981::blockSD::1163::Storage.Misc.excCmd::(mountMaster) /usr/bin/sudo -n /bin/chown vdsm:kvm /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,993::blockSD::1163::Storage.Misc.excCmd::(mountMaster) SUCCESS: <err> = ''; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:40,993::sd::647::Storage.StorageDomain::(createMasterTree) creating vms dir: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master/vms >Thread-114::DEBUG::2014-10-23 17:16:40,994::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:40,994::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:40,996::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:40,996::__init__::232::IOProcess::(_processLogs) (33) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:40,997::__init__::232::IOProcess::(_processLogs) (33) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:40,997::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:40,997::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:40,998::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,002::__init__::232::IOProcess::(_processLogs) (34) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,002::__init__::232::IOProcess::(_processLogs) (34) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,004::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,004::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,005::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,005::__init__::232::IOProcess::(_processLogs) (35) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,007::__init__::232::IOProcess::(_processLogs) (35) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,008::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,009::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,009::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,010::__init__::232::IOProcess::(_processLogs) (36) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,010::__init__::232::IOProcess::(_processLogs) (36) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,011::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,012::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,013::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,013::__init__::232::IOProcess::(_processLogs) (37) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,014::__init__::232::IOProcess::(_processLogs) (37) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,015::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,016::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,017::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,017::__init__::232::IOProcess::(_processLogs) (38) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,018::__init__::232::IOProcess::(_processLogs) (38) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,019::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,020::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,020::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,021::__init__::232::IOProcess::(_processLogs) (39) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,021::__init__::232::IOProcess::(_processLogs) (39) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,022::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,023::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,023::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,024::__init__::232::IOProcess::(_processLogs) (40) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,025::__init__::232::IOProcess::(_processLogs) (40) Queuing response >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,025::sd::650::Storage.StorageDomain::(createMasterTree) creating task dir: /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master/tasks >Thread-114::DEBUG::2014-10-23 17:16:41,027::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,027::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,028::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,029::__init__::232::IOProcess::(_processLogs) (41) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,030::__init__::232::IOProcess::(_processLogs) (41) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,031::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,032::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,033::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,033::__init__::232::IOProcess::(_processLogs) (42) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,034::__init__::232::IOProcess::(_processLogs) (42) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,036::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,037::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,037::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,038::__init__::232::IOProcess::(_processLogs) (43) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,039::__init__::232::IOProcess::(_processLogs) (43) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,042::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,043::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,043::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,044::__init__::232::IOProcess::(_processLogs) (44) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,045::__init__::232::IOProcess::(_processLogs) (44) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,046::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,047::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,047::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,048::__init__::232::IOProcess::(_processLogs) (45) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,049::__init__::232::IOProcess::(_processLogs) (45) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,051::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,052::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,052::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,053::__init__::232::IOProcess::(_processLogs) (46) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,053::__init__::232::IOProcess::(_processLogs) (46) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,056::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,056::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,057::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,058::__init__::232::IOProcess::(_processLogs) (47) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,058::__init__::232::IOProcess::(_processLogs) (47) Queuing response >Thread-114::DEBUG::2014-10-23 17:16:41,061::__init__::232::IOProcess::(_processLogs) Receiving request... >Thread-114::DEBUG::2014-10-23 17:16:41,061::__init__::232::IOProcess::(_processLogs) Queuing request in the thread pool... >Thread-114::DEBUG::2014-10-23 17:16:41,062::__init__::232::IOProcess::(_processLogs) Extracting request information... >Thread-114::DEBUG::2014-10-23 17:16:41,063::__init__::232::IOProcess::(_processLogs) (48) Got request for method 'mkdir' >Thread-114::DEBUG::2014-10-23 17:16:41,064::__init__::232::IOProcess::(_processLogs) (48) Queuing response >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,066::threadPool::35::Storage.ThreadPool::(__init__) Enter - numThreads: 5, waitTimeout: 3, maxTasks: 500 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,070::storage_mailbox::574::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor - clearing outgoing mail, command is: ['dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox', 'oflag=direct', 'iflag=fullblock', 'conv=notrunc', 'count=1'] >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,070::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/outbox oflag=direct iflag=fullblock conv=notrunc count=1 bs=1024000 (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,105::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0283009 s, 36.2 MB/s\n'; <rc> = 0 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,105::storage_mailbox::582::Storage.MailBox.SpmMailMonitor::(__init__) SPM_MailMonitor created for pool 00000002-0002-0002-0002-000000000067 >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,106::sp::319::Storage.StoragePool::(startSpm) SPM mailbox ready for pool 00000002-0002-0002-0002-000000000067 on master domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Dummy-255::DEBUG::2014-10-23 17:16:41,106::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) dd if=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=direct,fullblock count=1 bs=1024000 (cwd None) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,107::sp::327::Storage.StoragePool::(startSpm) ended. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,113::task::595::Storage.TaskManager.Task::(_updateState) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::moving from state running -> state finished >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,114::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,114::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,115::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,116::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,116::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,116::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,117::task::916::Storage.TaskManager.Task::(_runJobs) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::Task.run: exit - success: result >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,118::task::993::Storage.TaskManager.Task::(_decref) Task=`f2653993-1637-420a-9dfc-afcb1584c7ed`::ref 0 aborting False >f2653993-1637-420a-9dfc-afcb1584c7ed::DEBUG::2014-10-23 17:16:41,118::threadPool::57::Storage.ThreadPool::(setRunningTask) Number of running tasks: 0 >Dummy-255::DEBUG::2014-10-23 17:16:41,159::storage_mailbox::731::Storage.Misc.excCmd::(_checkForMail) SUCCESS: <err> = '1+0 records in\n1+0 records out\n1024000 bytes (1.0 MB) copied, 0.0191671 s, 53.4 MB/s\n'; <rc> = 0 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:41,780::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-256::DEBUG::2014-10-23 17:16:41,782::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.getStatus' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:41,782::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-256::DEBUG::2014-10-23 17:16:41,785::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cc2a95e-5787-459f-bc08-e1f33e695675`::moving from state init -> state preparing >Thread-256::INFO::2014-10-23 17:16:41,786::logUtils::44::dispatcher::(wrapper) Run and protect: getTaskStatus(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-256::DEBUG::2014-10-23 17:16:41,786::taskManager::103::Storage.TaskManager::(getTaskStatus) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-256::DEBUG::2014-10-23 17:16:41,787::taskManager::106::Storage.TaskManager::(getTaskStatus) Return. Response: {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-256::INFO::2014-10-23 17:16:41,787::logUtils::47::dispatcher::(wrapper) Run and protect: getTaskStatus, Return response: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-256::DEBUG::2014-10-23 17:16:41,787::task::1191::Storage.TaskManager.Task::(prepare) Task=`1cc2a95e-5787-459f-bc08-e1f33e695675`::finished: {'taskStatus': {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'}} >Thread-256::DEBUG::2014-10-23 17:16:41,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`1cc2a95e-5787-459f-bc08-e1f33e695675`::moving from state preparing -> state finished >Thread-256::DEBUG::2014-10-23 17:16:41,788::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-256::DEBUG::2014-10-23 17:16:41,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-256::DEBUG::2014-10-23 17:16:41,789::task::993::Storage.TaskManager.Task::(_decref) Task=`1cc2a95e-5787-459f-bc08-e1f33e695675`::ref 0 aborting False >Thread-256::DEBUG::2014-10-23 17:16:41,789::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.getStatus' in bridge with {'code': 0, 'message': '1 jobs completed successfully', 'taskState': 'finished', 'taskResult': 'success', 'taskID': 'f2653993-1637-420a-9dfc-afcb1584c7ed'} >Thread-256::DEBUG::2014-10-23 17:16:41,790::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:41,794::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-257::DEBUG::2014-10-23 17:16:41,796::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.getSpmStatus' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067'} >JsonRpcServer::DEBUG::2014-10-23 17:16:41,796::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-257::DEBUG::2014-10-23 17:16:41,799::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e579ac1-78b0-4759-a9e7-a7dfada27d2c`::moving from state init -> state preparing >Thread-257::INFO::2014-10-23 17:16:41,799::logUtils::44::dispatcher::(wrapper) Run and protect: getSpmStatus(spUUID=u'00000002-0002-0002-0002-000000000067', options=None) >Thread-257::INFO::2014-10-23 17:16:41,811::logUtils::47::dispatcher::(wrapper) Run and protect: getSpmStatus, Return response: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-257::DEBUG::2014-10-23 17:16:41,812::task::1191::Storage.TaskManager.Task::(prepare) Task=`3e579ac1-78b0-4759-a9e7-a7dfada27d2c`::finished: {'spm_st': {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L}} >Thread-257::DEBUG::2014-10-23 17:16:41,813::task::595::Storage.TaskManager.Task::(_updateState) Task=`3e579ac1-78b0-4759-a9e7-a7dfada27d2c`::moving from state preparing -> state finished >Thread-257::DEBUG::2014-10-23 17:16:41,813::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-257::DEBUG::2014-10-23 17:16:41,814::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-257::DEBUG::2014-10-23 17:16:41,815::task::993::Storage.TaskManager.Task::(_decref) Task=`3e579ac1-78b0-4759-a9e7-a7dfada27d2c`::ref 0 aborting False >Thread-257::DEBUG::2014-10-23 17:16:41,815::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.getSpmStatus' in bridge with {'spmId': 1, 'spmStatus': 'SPM', 'spmLver': 3L} >Thread-257::DEBUG::2014-10-23 17:16:41,816::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:41,821::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-258::DEBUG::2014-10-23 17:16:41,823::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'Task.clear' in bridge with {u'taskID': u'f2653993-1637-420a-9dfc-afcb1584c7ed'} >JsonRpcServer::DEBUG::2014-10-23 17:16:41,823::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-258::DEBUG::2014-10-23 17:16:41,826::task::595::Storage.TaskManager.Task::(_updateState) Task=`31760502-932d-4d11-ba8f-674c9c4b12ac`::moving from state init -> state preparing >Thread-258::INFO::2014-10-23 17:16:41,827::logUtils::44::dispatcher::(wrapper) Run and protect: clearTask(taskID=u'f2653993-1637-420a-9dfc-afcb1584c7ed', spUUID=None, options=None) >Thread-258::DEBUG::2014-10-23 17:16:41,827::taskManager::171::Storage.TaskManager::(clearTask) Entry. taskID: f2653993-1637-420a-9dfc-afcb1584c7ed >Thread-258::DEBUG::2014-10-23 17:16:41,827::taskManager::176::Storage.TaskManager::(clearTask) Return. >Thread-258::INFO::2014-10-23 17:16:41,828::logUtils::47::dispatcher::(wrapper) Run and protect: clearTask, Return response: None >Thread-258::DEBUG::2014-10-23 17:16:41,828::task::1191::Storage.TaskManager.Task::(prepare) Task=`31760502-932d-4d11-ba8f-674c9c4b12ac`::finished: None >Thread-258::DEBUG::2014-10-23 17:16:41,829::task::595::Storage.TaskManager.Task::(_updateState) Task=`31760502-932d-4d11-ba8f-674c9c4b12ac`::moving from state preparing -> state finished >Thread-258::DEBUG::2014-10-23 17:16:41,829::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-258::DEBUG::2014-10-23 17:16:41,830::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-258::DEBUG::2014-10-23 17:16:41,830::task::993::Storage.TaskManager.Task::(_decref) Task=`31760502-932d-4d11-ba8f-674c9c4b12ac`::ref 0 aborting False >Thread-258::DEBUG::2014-10-23 17:16:41,830::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'Task.clear' in bridge with True >Thread-258::DEBUG::2014-10-23 17:16:41,831::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Detector thread::DEBUG::2014-10-23 17:16:42,026::protocoldetector::166::vds.MultiProtocolAcceptor::(_add_connection) Adding connection from 10.34.63.25:44591 >Detector thread::DEBUG::2014-10-23 17:16:42,030::protocoldetector::177::vds.MultiProtocolAcceptor::(_remove_connection) Connection removed from 10.34.63.25:44591 >Detector thread::DEBUG::2014-10-23 17:16:42,030::protocoldetector::203::vds.MultiProtocolAcceptor::(_handle_connection_read) Detected protocol stomp from 10.34.63.25:44591 >Detector thread::DEBUG::2014-10-23 17:16:42,030::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='CONNECT'> >Detector thread::INFO::2014-10-23 17:16:42,030::stompReactor::66::Broker.StompAdapter::(_cmd_connect) Processing CONNECT request >Detector thread::DEBUG::2014-10-23 17:16:42,030::stompReactor::338::protocoldetector.StompDetector::(handleSocket) Stomp detected from ('10.34.63.25', 44591) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:42,031::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SUBSCRIBE'> >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:42,031::stompReactor::89::Broker.StompAdapter::(_cmd_subscribe) Subscribe command ignored >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:42,072::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:42,072::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-259::DEBUG::2014-10-23 17:16:42,072::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.destroy' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'hostID': 1} >Thread-259::DEBUG::2014-10-23 17:16:42,073::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::moving from state init -> state preparing >Thread-259::INFO::2014-10-23 17:16:42,073::logUtils::44::dispatcher::(wrapper) Run and protect: destroyStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, options=None) >Thread-259::INFO::2014-10-23 17:16:42,073::hsm::1153::Storage.HSM::(destroyStoragePool) spUUID=00000002-0002-0002-0002-000000000067 >Thread-259::DEBUG::2014-10-23 17:16:42,074::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`02176330-8cbe-4122-a8f7-ed92a32b3772`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1159' at 'destroyStoragePool' >Thread-259::DEBUG::2014-10-23 17:16:42,074::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-259::DEBUG::2014-10-23 17:16:42,074::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-259::DEBUG::2014-10-23 17:16:42,074::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.00000002-0002-0002-0002-000000000067`ReqID=`02176330-8cbe-4122-a8f7-ed92a32b3772`::Granted request >Thread-259::DEBUG::2014-10-23 17:16:42,074::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::_resourcesAcquired: Storage.00000002-0002-0002-0002-000000000067 (exclusive) >Thread-259::DEBUG::2014-10-23 17:16:42,074::task::993::Storage.TaskManager.Task::(_decref) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::ref 1 aborting False >Thread-259::DEBUG::2014-10-23 17:16:42,075::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`4da36cc5-7f9c-4f93-b99f-52a5b0454cdf`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1163' at 'destroyStoragePool' >Thread-259::DEBUG::2014-10-23 17:16:42,075::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' for lock type 'exclusive' >Thread-259::DEBUG::2014-10-23 17:16:42,075::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free. Now locking as 'exclusive' (1 active user) >Thread-259::DEBUG::2014-10-23 17:16:42,075::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f`ReqID=`4da36cc5-7f9c-4f93-b99f-52a5b0454cdf`::Granted request >Thread-259::DEBUG::2014-10-23 17:16:42,075::task::827::Storage.TaskManager.Task::(resourceAcquired) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::_resourcesAcquired: Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (exclusive) >Thread-259::DEBUG::2014-10-23 17:16:42,075::task::993::Storage.TaskManager.Task::(_decref) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::ref 1 aborting False >Thread-259::WARNING::2014-10-23 17:16:42,075::sp::927::Storage.StoragePool::(forcedDetachSD) Force detaching domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >Thread-259::INFO::2014-10-23 17:16:42,075::spbackends::433::Storage.StoragePoolMemoryBackend::(setDomainsMap) new storage pool master version 1 and domains map {} >Thread-259::DEBUG::2014-10-23 17:16:42,075::sp::1143::Storage.StoragePool::(_cleanupDomainLinks) Removing: /rhev/data-center/00000002-0002-0002-0002-000000000067/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-259::DEBUG::2014-10-23 17:16:42,076::sp::337::Storage.StoragePool::(_shutDownUpgrade) Shutting down upgrade process >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`9f467c30-96fe-42cd-a5b9-287b85f36ea5`::Request was made in '/usr/share/vdsm/storage/sp.py' line '339' at '_shutDownUpgrade' >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' for lock type 'exclusive' >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free. Now locking as 'exclusive' (1 active user) >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.upgrade_00000002-0002-0002-0002-000000000067`ReqID=`9f467c30-96fe-42cd-a5b9-287b85f36ea5`::Granted request >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' (0 active users) >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-259::DEBUG::2014-10-23 17:16:42,076::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.upgrade_00000002-0002-0002-0002-000000000067', Clearing records. >Thread-259::DEBUG::2014-10-23 17:16:42,078::sp::375::Storage.StoragePool::(cleanupMasterMount) unmounting /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master >Thread-259::DEBUG::2014-10-23 17:16:42,106::mount::227::Storage.Misc.excCmd::(_runcmd) /usr/bin/sudo -n /bin/umount /rhev/data-center/mnt/blockSD/5c8c20ff-41e1-4a43-91a8-fbffa2f8334f/master (cwd None) >Thread-259::INFO::2014-10-23 17:16:42,172::clusterlock::339::Storage.SANLock::(release) Releasing cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-259::DEBUG::2014-10-23 17:16:42,186::clusterlock::349::Storage.SANLock::(release) Cluster lock for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f successfully released >Thread-259::DEBUG::2014-10-23 17:16:42,186::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sp.updateMonitoringThreads) >Thread-259::DEBUG::2014-10-23 17:16:42,186::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-259::INFO::2014-10-23 17:16:42,186::domainMonitor::114::Storage.DomainMonitor::(stopMonitoring) Stop monitoring 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-209::DEBUG::2014-10-23 17:16:42,186::domainMonitor::192::Storage.DomainMonitorThread::(_monitorLoop) Stopping domain monitor for 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f >Thread-209::INFO::2014-10-23 17:16:42,186::clusterlock::245::Storage.SANLock::(releaseHostId) Releasing host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (id: 1) >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:42,358::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:42,359::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-260::DEBUG::2014-10-23 17:16:42,360::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Dummy-255::INFO::2014-10-23 17:16:44,069::storage_mailbox::792::Storage.MailBox.SpmMailMonitor::(run) SPM_MailMonitor - Incoming mail monitoring thread stopped >Thread-209::DEBUG::2014-10-23 17:16:44,187::clusterlock::255::Storage.SANLock::(releaseHostId) Host id for domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f released successfully (id: 1) >Thread-259::DEBUG::2014-10-23 17:16:44,187::sp::1407::Storage.StoragePool::(updateMonitoringThreads) Storage Pool `00000002-0002-0002-0002-000000000067` stopped monitoring domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` >Thread-259::DEBUG::2014-10-23 17:16:44,188::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-259::DEBUG::2014-10-23 17:16:44,188::sp::944::Storage.StoragePool::(forcedDetachSD) Force detach for domain `5c8c20ff-41e1-4a43-91a8-fbffa2f8334f` is done >Thread-259::INFO::2014-10-23 17:16:44,188::sd::533::Storage.StorageDomain::(detach) detaching storage domain 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f from pool 00000002-0002-0002-0002-000000000067 >Thread-259::DEBUG::2014-10-23 17:16:44,188::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,188::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,188::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,188::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,188::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,189::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-259::DEBUG::2014-10-23 17:16:44,283::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-259::DEBUG::2014-10-23 17:16:44,284::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,284::persistentDict::234::Storage.PersistentDict::(refresh) read lines (VGTagMetadataRW)=['LEASETIMESEC=60', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'VERSION=3', 'TYPE=ISCSI', 'DESCRIPTION=iscsi', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'LOGBLKSIZE=512', 'LEASERETRIES=3', 'IOOPTIMEOUTSEC=10', 'LOCKRENEWALINTERVALSEC=5', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', 'PHYBLKSIZE=512', 'CLASS=Data', 'LOCKPOLICY=', 'POOL_SPM_ID=-1', 'POOL_DESCRIPTION=Default', 'POOL_SPM_LVER=-1', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'MASTER_VERSION=1', 'ROLE=Master', '_SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7', 'POOL_UUID=00000002-0002-0002-0002-000000000067'] >Thread-259::DEBUG::2014-10-23 17:16:44,284::persistentDict::167::Storage.PersistentDict::(transaction) Starting transaction >Thread-259::DEBUG::2014-10-23 17:16:44,284::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-259::DEBUG::2014-10-23 17:16:44,284::persistentDict::160::Storage.PersistentDict::(transaction) Reusing active transaction >Thread-259::DEBUG::2014-10-23 17:16:44,284::persistentDict::173::Storage.PersistentDict::(transaction) Flushing changes >Thread-259::DEBUG::2014-10-23 17:16:44,285::persistentDict::305::Storage.PersistentDict::(flush) about to write lines (VGTagMetadataRW)=['CLASS=Data', 'DESCRIPTION=iscsi', 'IOOPTIMEOUTSEC=10', 'LEASERETRIES=3', 'LEASETIMESEC=60', 'LOCKPOLICY=', 'LOCKRENEWALINTERVALSEC=5', 'LOGBLKSIZE=512', 'MASTER_VERSION=1', 'PHYBLKSIZE=512', 'POOL_DESCRIPTION=Default', 'POOL_DOMAINS=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f:Active', 'POOL_SPM_ID=-1', 'POOL_SPM_LVER=-1', 'POOL_UUID=', u'PV0=pv:1cmg01,uuid:0nX55X-q6dP-w5yz-Iyel-3c6H-JT2v-ZEXfpC,pestart:0,pecount:677,mapoffset:0', 'ROLE=Regular', 'SDUUID=5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', 'TYPE=ISCSI', 'VERSION=3', 'VGUUID=L4kjm1-TTcv-3LdY-os6f-JImd-KOB2-hma7pP', '_SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef'] >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::505::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::517::Storage.OperationMutex::(_invalidatelvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::365::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,285::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgs --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|/dev/mapper/1cmg02|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --noheadings --units b --nosuffix --separator '|' --ignoreskippedcluster -o uuid,name,attr,size,free,extent_size,extent_count,free_count,tags,vg_mda_size,vg_mda_free,lv_count,pv_count,pv_name 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-259::DEBUG::2014-10-23 17:16:44,384::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-259::DEBUG::2014-10-23 17:16:44,385::lvm::407::Storage.OperationMutex::(_reloadvgs) Operation 'lvm reload operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,385::blockSD::315::Storage.Metadata.VGTagMetadataRW::(writelines) Updating metadata adding=MDT_ROLE=Regular, MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef, MDT_POOL_UUID= removing=MDT_ROLE=Master, MDT__SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7, MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 >Thread-259::DEBUG::2014-10-23 17:16:44,385::lvm::288::Storage.Misc.excCmd::(cmd) /usr/bin/sudo -n /sbin/lvm vgchange --config ' devices { preferred_names = ["^/dev/mapper/"] ignore_suspended_devices=1 write_cache_state=0 disable_after_error_count=3 obtain_device_list_from_udev=0 filter = [ '\''a|/dev/mapper/1cmg01|'\'', '\''r|.*|'\'' ] } global { locking_type=1 prioritise_write_locks=1 wait_for_locks=1 use_lvmetad=0 } backup { retain_min = 50 retain_days = 0 } ' --deltag MDT_ROLE=Master --deltag MDT__SHA_CKSUM=e515faeffdd7374e24565c6520fc827a84ea42f7 --deltag MDT_POOL_UUID=00000002-0002-0002-0002-000000000067 --addtag MDT__SHA_CKSUM=181af7bb17a9c322e46ab6cf13833cb1f22b29ef --addtag MDT_ROLE=Regular --addtag MDT_POOL_UUID= 5c8c20ff-41e1-4a43-91a8-fbffa2f8334f (cwd None) >Thread-259::DEBUG::2014-10-23 17:16:44,548::lvm::288::Storage.Misc.excCmd::(cmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-259::DEBUG::2014-10-23 17:16:44,548::lvm::495::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,548::lvm::497::Storage.OperationMutex::(_invalidatevgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-259::DEBUG::2014-10-23 17:16:44,549::persistentDict::175::Storage.PersistentDict::(transaction) Finished transaction >Thread-259::DEBUG::2014-10-23 17:16:44,549::resourceManager::198::Storage.ResourceManager.Request::(__init__) ResName=`Storage.HsmDomainMonitorLock`ReqID=`37e7d443-aee6-4e0d-8b71-43e1e6070a83`::Request was made in '/usr/share/vdsm/storage/hsm.py' line '1132' at '_disconnectPool' >Thread-259::DEBUG::2014-10-23 17:16:44,549::resourceManager::542::Storage.ResourceManager::(registerResource) Trying to register resource 'Storage.HsmDomainMonitorLock' for lock type 'exclusive' >Thread-259::DEBUG::2014-10-23 17:16:44,549::resourceManager::601::Storage.ResourceManager::(registerResource) Resource 'Storage.HsmDomainMonitorLock' is free. Now locking as 'exclusive' (1 active user) >Thread-259::DEBUG::2014-10-23 17:16:44,549::resourceManager::238::Storage.ResourceManager.Request::(grant) ResName=`Storage.HsmDomainMonitorLock`ReqID=`37e7d443-aee6-4e0d-8b71-43e1e6070a83`::Granted request >Thread-259::INFO::2014-10-23 17:16:44,550::sp::657::Storage.StoragePool::(disconnect) Disconnect from the storage pool 00000002-0002-0002-0002-000000000067 >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:45,396::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:45,398::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-261::DEBUG::2014-10-23 17:16:45,399::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >Thread-217::INFO::2014-10-23 17:16:45,490::storage_mailbox::509::Storage.MailBox.HsmMailMonitor::(run) HSM_MailboxMonitor - Incoming mail monitoring thread stopped, clearing outgoing mail >Thread-217::INFO::2014-10-23 17:16:45,490::storage_mailbox::385::Storage.MailBox.HsmMailMonitor::(_sendMail) HSM_MailMonitor sending mail to SPM - ['/bin/dd', 'of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox', 'iflag=fullblock', 'oflag=direct', 'conv=notrunc', 'bs=512', 'seek=8'] >Thread-217::DEBUG::2014-10-23 17:16:45,491::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) /bin/dd of=/rhev/data-center/00000002-0002-0002-0002-000000000067/mastersd/dom_md/inbox iflag=fullblock oflag=direct conv=notrunc bs=512 seek=8 (cwd None) >Thread-217::DEBUG::2014-10-23 17:16:45,498::storage_mailbox::84::Storage.Misc.excCmd::(_mboxExecCmd) SUCCESS: <err> = '8+0 records in\n8+0 records out\n4096 bytes (4.1 kB) copied, 0.00276493 s, 1.5 MB/s\n'; <rc> = 0 >Thread-259::DEBUG::2014-10-23 17:16:46,476::fileUtils::124::Storage.fileUtils::(cleanupdir) Removing directory: /rhev/data-center/00000002-0002-0002-0002-000000000067 >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.HsmDomainMonitorLock' >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.HsmDomainMonitorLock' (0 active users) >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.HsmDomainMonitorLock' is free, finding out if anyone is waiting for it. >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.HsmDomainMonitorLock', Clearing records. >Thread-259::INFO::2014-10-23 17:16:46,477::logUtils::47::dispatcher::(wrapper) Run and protect: destroyStoragePool, Return response: True >Thread-259::DEBUG::2014-10-23 17:16:46,477::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::finished: True >Thread-259::DEBUG::2014-10-23 17:16:46,477::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::moving from state preparing -> state finished >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {u'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f': < ResourceRef 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', isValid: 'True' obj: 'None'>, 'Storage.00000002-0002-0002-0002-000000000067': < ResourceRef 'Storage.00000002-0002-0002-0002-000000000067', isValid: 'True' obj: 'None'>} >Thread-259::DEBUG::2014-10-23 17:16:46,477::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' (0 active users) >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f' is free, finding out if anyone is waiting for it. >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.5c8c20ff-41e1-4a43-91a8-fbffa2f8334f', Clearing records. >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::616::Storage.ResourceManager::(releaseResource) Trying to release resource 'Storage.00000002-0002-0002-0002-000000000067' >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::635::Storage.ResourceManager::(releaseResource) Released resource 'Storage.00000002-0002-0002-0002-000000000067' (0 active users) >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::641::Storage.ResourceManager::(releaseResource) Resource 'Storage.00000002-0002-0002-0002-000000000067' is free, finding out if anyone is waiting for it. >Thread-259::DEBUG::2014-10-23 17:16:46,478::resourceManager::649::Storage.ResourceManager::(releaseResource) No one is waiting for resource 'Storage.00000002-0002-0002-0002-000000000067', Clearing records. >Thread-259::DEBUG::2014-10-23 17:16:46,479::task::993::Storage.TaskManager.Task::(_decref) Task=`0ea854a7-e309-4b87-a64b-f5cf4fba9816`::ref 0 aborting False >Thread-259::DEBUG::2014-10-23 17:16:46,479::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.destroy' in bridge with True >Thread-259::DEBUG::2014-10-23 17:16:46,479::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:46,540::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >Thread-262::DEBUG::2014-10-23 17:16:46,540::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnect' in bridge with {u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'scsiKey': u'00000002-0002-0002-0002-000000000067', u'hostID': 1} >JsonRpcServer::DEBUG::2014-10-23 17:16:46,540::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-262::DEBUG::2014-10-23 17:16:46,541::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e756f6-5de9-490a-8734-0f6c7a91180d`::moving from state init -> state preparing >Thread-262::INFO::2014-10-23 17:16:46,542::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStoragePool(spUUID=u'00000002-0002-0002-0002-000000000067', hostID=1, remove=False, options=None) >Thread-262::WARNING::2014-10-23 17:16:46,542::hsm::1119::Storage.HSM::(disconnectStoragePool) disconnect sp: 00000002-0002-0002-0002-000000000067 failed. Known pools {} >Thread-262::INFO::2014-10-23 17:16:46,542::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStoragePool, Return response: None >Thread-262::DEBUG::2014-10-23 17:16:46,542::task::1191::Storage.TaskManager.Task::(prepare) Task=`61e756f6-5de9-490a-8734-0f6c7a91180d`::finished: None >Thread-262::DEBUG::2014-10-23 17:16:46,542::task::595::Storage.TaskManager.Task::(_updateState) Task=`61e756f6-5de9-490a-8734-0f6c7a91180d`::moving from state preparing -> state finished >Thread-262::DEBUG::2014-10-23 17:16:46,542::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-262::DEBUG::2014-10-23 17:16:46,542::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-262::DEBUG::2014-10-23 17:16:46,542::task::993::Storage.TaskManager.Task::(_decref) Task=`61e756f6-5de9-490a-8734-0f6c7a91180d`::ref 0 aborting False >Thread-262::DEBUG::2014-10-23 17:16:46,542::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnect' in bridge with True >Thread-262::DEBUG::2014-10-23 17:16:46,542::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:46,571::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:46,572::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-263::DEBUG::2014-10-23 17:16:46,572::__init__::467::jsonrpc.JsonRpcServer::(_serveRequest) Calling 'StoragePool.disconnectStorageServer' in bridge with {u'connectionParams': [{u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': u'', u'port': u'3260'}], u'storagepoolID': u'00000002-0002-0002-0002-000000000067', u'domainType': 3} >Thread-263::DEBUG::2014-10-23 17:16:46,573::task::595::Storage.TaskManager.Task::(_updateState) Task=`97fe4605-ccfd-4fb7-a324-e89238a26739`::moving from state init -> state preparing >Thread-263::INFO::2014-10-23 17:16:46,573::logUtils::44::dispatcher::(wrapper) Run and protect: disconnectStorageServer(domType=3, spUUID=u'00000002-0002-0002-0002-000000000067', conList=[{u'connection': u'10.34.63.202', u'iqn': u'iqn.2012-10.brq.str-01:cmestreg', u'user': u'', u'tpgt': u'1', u'password': '******', u'id': u'84d26a30-778b-4513-aa05-508ece2dbae1', u'port': u'3260'}], options=None) >Thread-263::DEBUG::2014-10-23 17:16:46,581::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -r 5 -u (cwd None) >Thread-263::DEBUG::2014-10-23 17:16:47,165::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) SUCCESS: <err> = ''; <rc> = 0 >Thread-263::DEBUG::2014-10-23 17:16:47,165::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.sdc.refreshStorage) >Thread-263::DEBUG::2014-10-23 17:16:47,165::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-263::DEBUG::2014-10-23 17:16:47,165::misc::741::Storage.SamplingMethod::(__call__) Trying to enter sampling method (storage.iscsi.rescan) >Thread-263::DEBUG::2014-10-23 17:16:47,165::misc::743::Storage.SamplingMethod::(__call__) Got in to sampling method >Thread-263::DEBUG::2014-10-23 17:16:47,165::iscsi::403::Storage.ISCSI::(rescan) Performing SCSI scan, this will take up to 30 seconds >Thread-263::DEBUG::2014-10-23 17:16:47,166::iscsiadm::92::Storage.Misc.excCmd::(_runCmd) /usr/bin/sudo -n /sbin/iscsiadm -m session -R (cwd None) >Thread-263::DEBUG::2014-10-23 17:16:47,174::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-263::DEBUG::2014-10-23 17:16:47,175::multipath::113::Storage.Misc.excCmd::(rescan) /usr/bin/sudo -n /sbin/multipath (cwd None) >Thread-263::DEBUG::2014-10-23 17:16:47,279::multipath::113::Storage.Misc.excCmd::(rescan) SUCCESS: <err> = ''; <rc> = 0 >Thread-263::DEBUG::2014-10-23 17:16:47,279::lvm::489::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::lvm::491::Storage.OperationMutex::(_invalidateAllPvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::lvm::500::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' got the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::lvm::502::Storage.OperationMutex::(_invalidateAllVgs) Operation 'lvm invalidate operation' released the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::lvm::520::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' got the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::lvm::522::Storage.OperationMutex::(_invalidateAllLvs) Operation 'lvm invalidate operation' released the operation mutex >Thread-263::DEBUG::2014-10-23 17:16:47,280::misc::751::Storage.SamplingMethod::(__call__) Returning last result >Thread-263::INFO::2014-10-23 17:16:47,280::logUtils::47::dispatcher::(wrapper) Run and protect: disconnectStorageServer, Return response: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-263::DEBUG::2014-10-23 17:16:47,280::task::1191::Storage.TaskManager.Task::(prepare) Task=`97fe4605-ccfd-4fb7-a324-e89238a26739`::finished: {'statuslist': [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}]} >Thread-263::DEBUG::2014-10-23 17:16:47,281::task::595::Storage.TaskManager.Task::(_updateState) Task=`97fe4605-ccfd-4fb7-a324-e89238a26739`::moving from state preparing -> state finished >Thread-263::DEBUG::2014-10-23 17:16:47,281::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-263::DEBUG::2014-10-23 17:16:47,281::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-263::DEBUG::2014-10-23 17:16:47,281::task::993::Storage.TaskManager.Task::(_decref) Task=`97fe4605-ccfd-4fb7-a324-e89238a26739`::ref 0 aborting False >Thread-263::DEBUG::2014-10-23 17:16:47,281::__init__::498::jsonrpc.JsonRpcServer::(_serveRequest) Return 'StoragePool.disconnectStorageServer' in bridge with [{'status': 0, 'id': u'84d26a30-778b-4513-aa05-508ece2dbae1'}] >Thread-263::DEBUG::2014-10-23 17:16:47,281::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:48,428::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:48,429::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-264::DEBUG::2014-10-23 17:16:48,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:51,441::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:51,442::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-265::DEBUG::2014-10-23 17:16:51,443::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:54,453::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:54,454::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-266::DEBUG::2014-10-23 17:16:54,455::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb14521-8560-44a2-a555-4c9ccb94f79a`::moving from state init -> state preparing >Thread-266::INFO::2014-10-23 17:16:54,455::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-266::INFO::2014-10-23 17:16:54,456::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-266::DEBUG::2014-10-23 17:16:54,456::task::1191::Storage.TaskManager.Task::(prepare) Task=`1eb14521-8560-44a2-a555-4c9ccb94f79a`::finished: {} >Thread-266::DEBUG::2014-10-23 17:16:54,456::task::595::Storage.TaskManager.Task::(_updateState) Task=`1eb14521-8560-44a2-a555-4c9ccb94f79a`::moving from state preparing -> state finished >Thread-266::DEBUG::2014-10-23 17:16:54,456::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-266::DEBUG::2014-10-23 17:16:54,456::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-266::DEBUG::2014-10-23 17:16:54,456::task::993::Storage.TaskManager.Task::(_decref) Task=`1eb14521-8560-44a2-a555-4c9ccb94f79a`::ref 0 aborting False >Thread-266::DEBUG::2014-10-23 17:16:54,458::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:54,493::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:54,494::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-267::DEBUG::2014-10-23 17:16:54,495::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:16:57,697::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:16:57,698::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-268::DEBUG::2014-10-23 17:16:57,698::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:00,710::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:00,711::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-269::DEBUG::2014-10-23 17:17:00,712::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:03,724::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:03,725::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-270::DEBUG::2014-10-23 17:17:03,726::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:06,756::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:06,757::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-271::DEBUG::2014-10-23 17:17:06,758::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:09,773::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:09,774::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-272::DEBUG::2014-10-23 17:17:09,775::task::595::Storage.TaskManager.Task::(_updateState) Task=`f27b8b3d-a36f-41ec-be47-ece3e4781f3a`::moving from state init -> state preparing >Thread-272::INFO::2014-10-23 17:17:09,775::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-272::INFO::2014-10-23 17:17:09,775::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-272::DEBUG::2014-10-23 17:17:09,775::task::1191::Storage.TaskManager.Task::(prepare) Task=`f27b8b3d-a36f-41ec-be47-ece3e4781f3a`::finished: {} >Thread-272::DEBUG::2014-10-23 17:17:09,776::task::595::Storage.TaskManager.Task::(_updateState) Task=`f27b8b3d-a36f-41ec-be47-ece3e4781f3a`::moving from state preparing -> state finished >Thread-272::DEBUG::2014-10-23 17:17:09,776::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-272::DEBUG::2014-10-23 17:17:09,776::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-272::DEBUG::2014-10-23 17:17:09,776::task::993::Storage.TaskManager.Task::(_decref) Task=`f27b8b3d-a36f-41ec-be47-ece3e4781f3a`::ref 0 aborting False >Thread-272::DEBUG::2014-10-23 17:17:09,777::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:09,799::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:09,800::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-273::DEBUG::2014-10-23 17:17:09,801::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:12,912::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:12,912::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-274::DEBUG::2014-10-23 17:17:12,913::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:15,924::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:15,925::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-275::DEBUG::2014-10-23 17:17:15,926::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:18,937::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:18,938::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-276::DEBUG::2014-10-23 17:17:18,939::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:21,951::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:21,952::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-277::DEBUG::2014-10-23 17:17:21,952::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:24,963::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:24,964::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-278::DEBUG::2014-10-23 17:17:24,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`af6b2fd3-8b6e-4228-b2cc-9c6faeaea592`::moving from state init -> state preparing >Thread-278::INFO::2014-10-23 17:17:24,966::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-278::INFO::2014-10-23 17:17:24,966::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-278::DEBUG::2014-10-23 17:17:24,966::task::1191::Storage.TaskManager.Task::(prepare) Task=`af6b2fd3-8b6e-4228-b2cc-9c6faeaea592`::finished: {} >Thread-278::DEBUG::2014-10-23 17:17:24,966::task::595::Storage.TaskManager.Task::(_updateState) Task=`af6b2fd3-8b6e-4228-b2cc-9c6faeaea592`::moving from state preparing -> state finished >Thread-278::DEBUG::2014-10-23 17:17:24,966::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-278::DEBUG::2014-10-23 17:17:24,966::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-278::DEBUG::2014-10-23 17:17:24,966::task::993::Storage.TaskManager.Task::(_decref) Task=`af6b2fd3-8b6e-4228-b2cc-9c6faeaea592`::ref 0 aborting False >Thread-278::DEBUG::2014-10-23 17:17:24,968::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:24,995::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:24,996::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-279::DEBUG::2014-10-23 17:17:24,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:28,134::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:28,135::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-280::DEBUG::2014-10-23 17:17:28,136::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:31,148::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:31,149::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-281::DEBUG::2014-10-23 17:17:31,150::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:34,163::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:34,164::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-282::DEBUG::2014-10-23 17:17:34,165::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:37,186::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:37,187::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-283::DEBUG::2014-10-23 17:17:37,188::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:40,213::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:40,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-284::DEBUG::2014-10-23 17:17:40,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`27a8a853-ee50-4a3e-86cb-3872691dcdf4`::moving from state init -> state preparing >Thread-284::INFO::2014-10-23 17:17:40,215::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-284::INFO::2014-10-23 17:17:40,215::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-284::DEBUG::2014-10-23 17:17:40,215::task::1191::Storage.TaskManager.Task::(prepare) Task=`27a8a853-ee50-4a3e-86cb-3872691dcdf4`::finished: {} >Thread-284::DEBUG::2014-10-23 17:17:40,215::task::595::Storage.TaskManager.Task::(_updateState) Task=`27a8a853-ee50-4a3e-86cb-3872691dcdf4`::moving from state preparing -> state finished >Thread-284::DEBUG::2014-10-23 17:17:40,215::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-284::DEBUG::2014-10-23 17:17:40,215::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-284::DEBUG::2014-10-23 17:17:40,216::task::993::Storage.TaskManager.Task::(_decref) Task=`27a8a853-ee50-4a3e-86cb-3872691dcdf4`::ref 0 aborting False >Thread-284::DEBUG::2014-10-23 17:17:40,217::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:40,246::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:40,246::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-285::DEBUG::2014-10-23 17:17:40,247::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:43,374::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:43,375::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-286::DEBUG::2014-10-23 17:17:43,376::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:46,390::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:46,390::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-287::DEBUG::2014-10-23 17:17:46,391::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:49,405::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:49,406::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-288::DEBUG::2014-10-23 17:17:49,407::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:52,428::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:52,429::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-289::DEBUG::2014-10-23 17:17:52,430::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:55,443::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:55,444::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-290::DEBUG::2014-10-23 17:17:55,445::task::595::Storage.TaskManager.Task::(_updateState) Task=`d66cc9bb-aa12-46b2-9222-6f7bc3edd21d`::moving from state init -> state preparing >Thread-290::INFO::2014-10-23 17:17:55,445::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-290::INFO::2014-10-23 17:17:55,445::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-290::DEBUG::2014-10-23 17:17:55,446::task::1191::Storage.TaskManager.Task::(prepare) Task=`d66cc9bb-aa12-46b2-9222-6f7bc3edd21d`::finished: {} >Thread-290::DEBUG::2014-10-23 17:17:55,446::task::595::Storage.TaskManager.Task::(_updateState) Task=`d66cc9bb-aa12-46b2-9222-6f7bc3edd21d`::moving from state preparing -> state finished >Thread-290::DEBUG::2014-10-23 17:17:55,446::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-290::DEBUG::2014-10-23 17:17:55,446::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-290::DEBUG::2014-10-23 17:17:55,446::task::993::Storage.TaskManager.Task::(_decref) Task=`d66cc9bb-aa12-46b2-9222-6f7bc3edd21d`::ref 0 aborting False >Thread-290::DEBUG::2014-10-23 17:17:55,447::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:55,501::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:55,502::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-291::DEBUG::2014-10-23 17:17:55,503::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:17:58,689::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:17:58,690::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-292::DEBUG::2014-10-23 17:17:58,691::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:01,703::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:01,703::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-293::DEBUG::2014-10-23 17:18:01,704::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:04,717::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:04,718::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-294::DEBUG::2014-10-23 17:18:04,719::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:07,749::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:07,750::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-295::DEBUG::2014-10-23 17:18:07,750::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:10,785::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:10,786::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-296::DEBUG::2014-10-23 17:18:10,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`7815edde-46a0-4fe9-abf3-a28687e85faa`::moving from state init -> state preparing >Thread-296::INFO::2014-10-23 17:18:10,788::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-296::INFO::2014-10-23 17:18:10,788::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-296::DEBUG::2014-10-23 17:18:10,788::task::1191::Storage.TaskManager.Task::(prepare) Task=`7815edde-46a0-4fe9-abf3-a28687e85faa`::finished: {} >Thread-296::DEBUG::2014-10-23 17:18:10,788::task::595::Storage.TaskManager.Task::(_updateState) Task=`7815edde-46a0-4fe9-abf3-a28687e85faa`::moving from state preparing -> state finished >Thread-296::DEBUG::2014-10-23 17:18:10,788::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-296::DEBUG::2014-10-23 17:18:10,788::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-296::DEBUG::2014-10-23 17:18:10,788::task::993::Storage.TaskManager.Task::(_decref) Task=`7815edde-46a0-4fe9-abf3-a28687e85faa`::ref 0 aborting False >Thread-296::DEBUG::2014-10-23 17:18:10,790::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:10,809::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:10,810::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-297::DEBUG::2014-10-23 17:18:10,811::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:13,971::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:13,972::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-298::DEBUG::2014-10-23 17:18:13,973::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:17,024::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:17,025::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-299::DEBUG::2014-10-23 17:18:17,026::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:20,040::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:20,041::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-300::DEBUG::2014-10-23 17:18:20,042::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:23,061::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:23,061::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-301::DEBUG::2014-10-23 17:18:23,062::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:26,073::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:26,074::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-302::DEBUG::2014-10-23 17:18:26,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea0d98f5-4fd8-481c-b16d-fe72852dde90`::moving from state init -> state preparing >Thread-302::INFO::2014-10-23 17:18:26,075::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-302::INFO::2014-10-23 17:18:26,075::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-302::DEBUG::2014-10-23 17:18:26,075::task::1191::Storage.TaskManager.Task::(prepare) Task=`ea0d98f5-4fd8-481c-b16d-fe72852dde90`::finished: {} >Thread-302::DEBUG::2014-10-23 17:18:26,075::task::595::Storage.TaskManager.Task::(_updateState) Task=`ea0d98f5-4fd8-481c-b16d-fe72852dde90`::moving from state preparing -> state finished >Thread-302::DEBUG::2014-10-23 17:18:26,075::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-302::DEBUG::2014-10-23 17:18:26,076::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-302::DEBUG::2014-10-23 17:18:26,076::task::993::Storage.TaskManager.Task::(_decref) Task=`ea0d98f5-4fd8-481c-b16d-fe72852dde90`::ref 0 aborting False >Thread-302::DEBUG::2014-10-23 17:18:26,077::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:26,100::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:26,101::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-303::DEBUG::2014-10-23 17:18:26,102::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:29,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:29,233::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-304::DEBUG::2014-10-23 17:18:29,234::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:32,262::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:32,262::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-305::DEBUG::2014-10-23 17:18:32,263::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:35,276::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:35,277::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-306::DEBUG::2014-10-23 17:18:35,278::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:38,307::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:38,307::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-307::DEBUG::2014-10-23 17:18:38,308::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:41,328::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:41,331::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-308::DEBUG::2014-10-23 17:18:41,335::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ccbaadb-b062-459e-be3c-8f410a2e4433`::moving from state init -> state preparing >Thread-308::INFO::2014-10-23 17:18:41,336::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-308::INFO::2014-10-23 17:18:41,337::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-308::DEBUG::2014-10-23 17:18:41,338::task::1191::Storage.TaskManager.Task::(prepare) Task=`0ccbaadb-b062-459e-be3c-8f410a2e4433`::finished: {} >Thread-308::DEBUG::2014-10-23 17:18:41,338::task::595::Storage.TaskManager.Task::(_updateState) Task=`0ccbaadb-b062-459e-be3c-8f410a2e4433`::moving from state preparing -> state finished >Thread-308::DEBUG::2014-10-23 17:18:41,339::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-308::DEBUG::2014-10-23 17:18:41,339::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-308::DEBUG::2014-10-23 17:18:41,340::task::993::Storage.TaskManager.Task::(_decref) Task=`0ccbaadb-b062-459e-be3c-8f410a2e4433`::ref 0 aborting False >Thread-308::DEBUG::2014-10-23 17:18:41,357::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:41,393::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:41,394::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-309::DEBUG::2014-10-23 17:18:41,397::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:44,706::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:44,707::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-310::DEBUG::2014-10-23 17:18:44,708::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:47,725::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:47,726::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-311::DEBUG::2014-10-23 17:18:47,726::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:50,738::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:50,739::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-312::DEBUG::2014-10-23 17:18:50,740::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:53,781::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:53,781::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-313::DEBUG::2014-10-23 17:18:53,782::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:56,800::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:56,801::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-314::DEBUG::2014-10-23 17:18:56,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2bb3613-5ac3-40d7-b0d2-8c5586bafb9c`::moving from state init -> state preparing >Thread-314::INFO::2014-10-23 17:18:56,802::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-314::INFO::2014-10-23 17:18:56,802::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-314::DEBUG::2014-10-23 17:18:56,802::task::1191::Storage.TaskManager.Task::(prepare) Task=`b2bb3613-5ac3-40d7-b0d2-8c5586bafb9c`::finished: {} >Thread-314::DEBUG::2014-10-23 17:18:56,802::task::595::Storage.TaskManager.Task::(_updateState) Task=`b2bb3613-5ac3-40d7-b0d2-8c5586bafb9c`::moving from state preparing -> state finished >Thread-314::DEBUG::2014-10-23 17:18:56,803::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-314::DEBUG::2014-10-23 17:18:56,803::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-314::DEBUG::2014-10-23 17:18:56,803::task::993::Storage.TaskManager.Task::(_decref) Task=`b2bb3613-5ac3-40d7-b0d2-8c5586bafb9c`::ref 0 aborting False >Thread-314::DEBUG::2014-10-23 17:18:56,804::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:56,848::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:56,848::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-315::DEBUG::2014-10-23 17:18:56,849::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:18:59,959::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:18:59,960::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-316::DEBUG::2014-10-23 17:18:59,961::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:02,977::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:02,977::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-317::DEBUG::2014-10-23 17:19:02,978::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:05,994::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:05,995::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-318::DEBUG::2014-10-23 17:19:05,996::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:09,007::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:09,008::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-319::DEBUG::2014-10-23 17:19:09,010::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:12,030::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:12,031::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-320::DEBUG::2014-10-23 17:19:12,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c72ce86-4789-4fcd-8f1a-5edbe1d3fe5b`::moving from state init -> state preparing >Thread-320::INFO::2014-10-23 17:19:12,032::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-320::INFO::2014-10-23 17:19:12,032::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-320::DEBUG::2014-10-23 17:19:12,032::task::1191::Storage.TaskManager.Task::(prepare) Task=`8c72ce86-4789-4fcd-8f1a-5edbe1d3fe5b`::finished: {} >Thread-320::DEBUG::2014-10-23 17:19:12,032::task::595::Storage.TaskManager.Task::(_updateState) Task=`8c72ce86-4789-4fcd-8f1a-5edbe1d3fe5b`::moving from state preparing -> state finished >Thread-320::DEBUG::2014-10-23 17:19:12,032::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-320::DEBUG::2014-10-23 17:19:12,032::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-320::DEBUG::2014-10-23 17:19:12,033::task::993::Storage.TaskManager.Task::(_decref) Task=`8c72ce86-4789-4fcd-8f1a-5edbe1d3fe5b`::ref 0 aborting False >Thread-320::DEBUG::2014-10-23 17:19:12,034::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:12,116::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:12,117::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-321::DEBUG::2014-10-23 17:19:12,118::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:15,232::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:15,233::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-322::DEBUG::2014-10-23 17:19:15,235::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:18,250::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:18,251::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-323::DEBUG::2014-10-23 17:19:18,252::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:21,269::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:21,271::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-324::DEBUG::2014-10-23 17:19:21,273::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:24,298::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:24,299::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-325::DEBUG::2014-10-23 17:19:24,300::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:27,310::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:27,311::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-326::DEBUG::2014-10-23 17:19:27,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`880a3e8e-d669-4905-97f7-817b2e9b97ec`::moving from state init -> state preparing >Thread-326::INFO::2014-10-23 17:19:27,312::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-326::INFO::2014-10-23 17:19:27,312::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-326::DEBUG::2014-10-23 17:19:27,312::task::1191::Storage.TaskManager.Task::(prepare) Task=`880a3e8e-d669-4905-97f7-817b2e9b97ec`::finished: {} >Thread-326::DEBUG::2014-10-23 17:19:27,312::task::595::Storage.TaskManager.Task::(_updateState) Task=`880a3e8e-d669-4905-97f7-817b2e9b97ec`::moving from state preparing -> state finished >Thread-326::DEBUG::2014-10-23 17:19:27,312::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-326::DEBUG::2014-10-23 17:19:27,312::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-326::DEBUG::2014-10-23 17:19:27,312::task::993::Storage.TaskManager.Task::(_decref) Task=`880a3e8e-d669-4905-97f7-817b2e9b97ec`::ref 0 aborting False >Thread-326::DEBUG::2014-10-23 17:19:27,314::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:27,333::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:27,334::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-327::DEBUG::2014-10-23 17:19:27,335::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:30,453::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:30,454::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-328::DEBUG::2014-10-23 17:19:30,455::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:33,474::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:33,475::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-329::DEBUG::2014-10-23 17:19:33,475::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:36,487::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:36,488::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-330::DEBUG::2014-10-23 17:19:36,489::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:39,501::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:39,502::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-331::DEBUG::2014-10-23 17:19:39,503::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:42,514::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:42,515::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-332::DEBUG::2014-10-23 17:19:42,521::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d50dc32-f111-4fb0-9f84-cf5432ea7841`::moving from state init -> state preparing >Thread-332::INFO::2014-10-23 17:19:42,521::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-332::INFO::2014-10-23 17:19:42,522::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-332::DEBUG::2014-10-23 17:19:42,523::task::1191::Storage.TaskManager.Task::(prepare) Task=`2d50dc32-f111-4fb0-9f84-cf5432ea7841`::finished: {} >Thread-332::DEBUG::2014-10-23 17:19:42,523::task::595::Storage.TaskManager.Task::(_updateState) Task=`2d50dc32-f111-4fb0-9f84-cf5432ea7841`::moving from state preparing -> state finished >Thread-332::DEBUG::2014-10-23 17:19:42,524::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-332::DEBUG::2014-10-23 17:19:42,525::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-332::DEBUG::2014-10-23 17:19:42,525::task::993::Storage.TaskManager.Task::(_decref) Task=`2d50dc32-f111-4fb0-9f84-cf5432ea7841`::ref 0 aborting False >Thread-332::DEBUG::2014-10-23 17:19:42,527::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:42,544::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:42,545::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-333::DEBUG::2014-10-23 17:19:42,545::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:45,712::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:45,713::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-334::DEBUG::2014-10-23 17:19:45,714::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:48,724::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:48,725::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-335::DEBUG::2014-10-23 17:19:48,727::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:51,747::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:51,748::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-336::DEBUG::2014-10-23 17:19:51,749::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:54,760::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:54,761::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-337::DEBUG::2014-10-23 17:19:54,762::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:57,775::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:57,776::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-338::DEBUG::2014-10-23 17:19:57,777::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9cf2b07-f031-4861-a363-a8b010100502`::moving from state init -> state preparing >Thread-338::INFO::2014-10-23 17:19:57,777::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-338::INFO::2014-10-23 17:19:57,777::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-338::DEBUG::2014-10-23 17:19:57,777::task::1191::Storage.TaskManager.Task::(prepare) Task=`b9cf2b07-f031-4861-a363-a8b010100502`::finished: {} >Thread-338::DEBUG::2014-10-23 17:19:57,778::task::595::Storage.TaskManager.Task::(_updateState) Task=`b9cf2b07-f031-4861-a363-a8b010100502`::moving from state preparing -> state finished >Thread-338::DEBUG::2014-10-23 17:19:57,778::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-338::DEBUG::2014-10-23 17:19:57,778::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-338::DEBUG::2014-10-23 17:19:57,778::task::993::Storage.TaskManager.Task::(_decref) Task=`b9cf2b07-f031-4861-a363-a8b010100502`::ref 0 aborting False >Thread-338::DEBUG::2014-10-23 17:19:57,779::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:19:57,797::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:19:57,798::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-339::DEBUG::2014-10-23 17:19:57,799::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:00,928::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:00,929::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-340::DEBUG::2014-10-23 17:20:00,930::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:03,944::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:03,950::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-341::DEBUG::2014-10-23 17:20:03,951::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:06,970::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:06,971::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-342::DEBUG::2014-10-23 17:20:06,972::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:09,984::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:09,985::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-343::DEBUG::2014-10-23 17:20:09,986::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:12,998::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:12,998::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-344::DEBUG::2014-10-23 17:20:12,999::task::595::Storage.TaskManager.Task::(_updateState) Task=`919386c2-811f-4778-8199-b0144a68afd0`::moving from state init -> state preparing >Thread-344::INFO::2014-10-23 17:20:12,999::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-344::INFO::2014-10-23 17:20:12,999::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-344::DEBUG::2014-10-23 17:20:13,000::task::1191::Storage.TaskManager.Task::(prepare) Task=`919386c2-811f-4778-8199-b0144a68afd0`::finished: {} >Thread-344::DEBUG::2014-10-23 17:20:13,000::task::595::Storage.TaskManager.Task::(_updateState) Task=`919386c2-811f-4778-8199-b0144a68afd0`::moving from state preparing -> state finished >Thread-344::DEBUG::2014-10-23 17:20:13,000::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-344::DEBUG::2014-10-23 17:20:13,000::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-344::DEBUG::2014-10-23 17:20:13,000::task::993::Storage.TaskManager.Task::(_decref) Task=`919386c2-811f-4778-8199-b0144a68afd0`::ref 0 aborting False >Thread-344::DEBUG::2014-10-23 17:20:13,001::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:13,045::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:13,045::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-345::DEBUG::2014-10-23 17:20:13,046::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:16,178::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:16,179::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-346::DEBUG::2014-10-23 17:20:16,180::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:19,192::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:19,193::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-347::DEBUG::2014-10-23 17:20:19,195::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:22,214::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:22,214::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-348::DEBUG::2014-10-23 17:20:22,215::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:25,236::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:25,237::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-349::DEBUG::2014-10-23 17:20:25,238::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:28,254::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:28,255::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-350::DEBUG::2014-10-23 17:20:28,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dbc07a7-f1c4-49b1-8acf-a823bd9cc3bd`::moving from state init -> state preparing >Thread-350::INFO::2014-10-23 17:20:28,256::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-350::INFO::2014-10-23 17:20:28,256::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-350::DEBUG::2014-10-23 17:20:28,256::task::1191::Storage.TaskManager.Task::(prepare) Task=`5dbc07a7-f1c4-49b1-8acf-a823bd9cc3bd`::finished: {} >Thread-350::DEBUG::2014-10-23 17:20:28,256::task::595::Storage.TaskManager.Task::(_updateState) Task=`5dbc07a7-f1c4-49b1-8acf-a823bd9cc3bd`::moving from state preparing -> state finished >Thread-350::DEBUG::2014-10-23 17:20:28,256::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-350::DEBUG::2014-10-23 17:20:28,256::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-350::DEBUG::2014-10-23 17:20:28,256::task::993::Storage.TaskManager.Task::(_decref) Task=`5dbc07a7-f1c4-49b1-8acf-a823bd9cc3bd`::ref 0 aborting False >Thread-350::DEBUG::2014-10-23 17:20:28,258::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:28,290::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:28,290::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-351::DEBUG::2014-10-23 17:20:28,291::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:31,424::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:31,425::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-352::DEBUG::2014-10-23 17:20:31,426::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:34,436::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:34,437::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-353::DEBUG::2014-10-23 17:20:34,438::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:37,451::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:37,452::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-354::DEBUG::2014-10-23 17:20:37,452::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:40,464::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:40,465::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-355::DEBUG::2014-10-23 17:20:40,466::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:43,478::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:43,478::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-356::DEBUG::2014-10-23 17:20:43,479::task::595::Storage.TaskManager.Task::(_updateState) Task=`545e0082-8477-4ff1-83e7-bab48829e769`::moving from state init -> state preparing >Thread-356::INFO::2014-10-23 17:20:43,479::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-356::INFO::2014-10-23 17:20:43,480::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-356::DEBUG::2014-10-23 17:20:43,480::task::1191::Storage.TaskManager.Task::(prepare) Task=`545e0082-8477-4ff1-83e7-bab48829e769`::finished: {} >Thread-356::DEBUG::2014-10-23 17:20:43,480::task::595::Storage.TaskManager.Task::(_updateState) Task=`545e0082-8477-4ff1-83e7-bab48829e769`::moving from state preparing -> state finished >Thread-356::DEBUG::2014-10-23 17:20:43,480::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-356::DEBUG::2014-10-23 17:20:43,480::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-356::DEBUG::2014-10-23 17:20:43,480::task::993::Storage.TaskManager.Task::(_decref) Task=`545e0082-8477-4ff1-83e7-bab48829e769`::ref 0 aborting False >Thread-356::DEBUG::2014-10-23 17:20:43,482::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:43,511::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:43,512::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-357::DEBUG::2014-10-23 17:20:43,513::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:46,637::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:46,638::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-358::DEBUG::2014-10-23 17:20:46,639::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:49,651::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:49,651::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-359::DEBUG::2014-10-23 17:20:49,653::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:52,671::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:52,671::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-360::DEBUG::2014-10-23 17:20:52,672::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:55,696::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:55,697::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-361::DEBUG::2014-10-23 17:20:55,698::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:58,715::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:58,716::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-362::DEBUG::2014-10-23 17:20:58,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`93272e63-5c3f-448e-b2ea-942ea410c834`::moving from state init -> state preparing >Thread-362::INFO::2014-10-23 17:20:58,717::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-362::INFO::2014-10-23 17:20:58,717::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-362::DEBUG::2014-10-23 17:20:58,717::task::1191::Storage.TaskManager.Task::(prepare) Task=`93272e63-5c3f-448e-b2ea-942ea410c834`::finished: {} >Thread-362::DEBUG::2014-10-23 17:20:58,717::task::595::Storage.TaskManager.Task::(_updateState) Task=`93272e63-5c3f-448e-b2ea-942ea410c834`::moving from state preparing -> state finished >Thread-362::DEBUG::2014-10-23 17:20:58,717::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-362::DEBUG::2014-10-23 17:20:58,717::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-362::DEBUG::2014-10-23 17:20:58,717::task::993::Storage.TaskManager.Task::(_decref) Task=`93272e63-5c3f-448e-b2ea-942ea410c834`::ref 0 aborting False >Thread-362::DEBUG::2014-10-23 17:20:58,719::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:20:58,742::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:20:58,743::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-363::DEBUG::2014-10-23 17:20:58,744::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:01,930::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:01,931::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-364::DEBUG::2014-10-23 17:21:01,932::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:04,957::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:04,958::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-365::DEBUG::2014-10-23 17:21:04,959::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:08,005::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:08,006::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-366::DEBUG::2014-10-23 17:21:08,007::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:11,019::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:11,020::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-367::DEBUG::2014-10-23 17:21:11,021::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:14,108::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:14,109::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-368::DEBUG::2014-10-23 17:21:14,110::task::595::Storage.TaskManager.Task::(_updateState) Task=`33b2c7cb-558e-4914-931a-f0d913357d6b`::moving from state init -> state preparing >Thread-368::INFO::2014-10-23 17:21:14,110::logUtils::44::dispatcher::(wrapper) Run and protect: repoStats(options=None) >Thread-368::INFO::2014-10-23 17:21:14,110::logUtils::47::dispatcher::(wrapper) Run and protect: repoStats, Return response: {} >Thread-368::DEBUG::2014-10-23 17:21:14,111::task::1191::Storage.TaskManager.Task::(prepare) Task=`33b2c7cb-558e-4914-931a-f0d913357d6b`::finished: {} >Thread-368::DEBUG::2014-10-23 17:21:14,111::task::595::Storage.TaskManager.Task::(_updateState) Task=`33b2c7cb-558e-4914-931a-f0d913357d6b`::moving from state preparing -> state finished >Thread-368::DEBUG::2014-10-23 17:21:14,111::resourceManager::940::Storage.ResourceManager.Owner::(releaseAll) Owner.releaseAll requests {} resources {} >Thread-368::DEBUG::2014-10-23 17:21:14,111::resourceManager::977::Storage.ResourceManager.Owner::(cancelAll) Owner.cancelAll requests {} >Thread-368::DEBUG::2014-10-23 17:21:14,111::task::993::Storage.TaskManager.Task::(_decref) Task=`33b2c7cb-558e-4914-931a-f0d913357d6b`::ref 0 aborting False >Thread-368::DEBUG::2014-10-23 17:21:14,112::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:14,219::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:14,220::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-369::DEBUG::2014-10-23 17:21:14,220::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:17,473::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:17,474::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-370::DEBUG::2014-10-23 17:21:17,475::stompReactor::163::yajsonrpc.StompServer::(send) Sending response >JsonRpc (StompReactor)::DEBUG::2014-10-23 17:21:20,537::stompReactor::98::Broker.StompAdapter::(handle_frame) Handling message <StompFrame command='SEND'> >JsonRpcServer::DEBUG::2014-10-23 17:21:20,538::__init__::504::jsonrpc.JsonRpcServer::(serve_requests) Waiting for request >Thread-371::DEBUG::2014-10-23 17:21:20,539::stompReactor::163::yajsonrpc.StompServer::(send) Sending response
You cannot view the attachment while viewing its details because your browser does not support IFRAMEs.
View the attachment on a separate page
.
View Attachment As Raw
Actions:
View
Attachments on
bug 1156119
:
949975
| 949986